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ABSTRACT 

This report describes the "Differential Coursewor)t 
Patterns (DCP) Project," which developed the Cluster Analytic Model 
(CAM) to look at effects of different patterns of college coursework 
on the general learned abilities of students as determined by such 
measures as the Graduate Record Examination (GRE) General Test. The 
model looks for correlations between courses taken and student 
achievement on the nine item types of th« GRE, after the predicted 
effect of initial Scholastic Aptitude Test scores is removed. This 
study looked at courses taken by two samples of students at Clayton 
State College (Georgia) and Georgia State University. The model's 
value is discussed in terms of its usefulness in assessment, 
curriculum development, and student advising. Among general 
conclusions were the following $ approximately 90% of courses could lae 
accurately grouped according to differential effects on the general 
learned abilities of students; the development of general learned 
abilities did not have an exact one-to-one relationship with 
departmental categories; the development of general learned abilities 
was not confined to the lower division; and there was little formal 
monitoring and description of the curriculum in terms of general 
learned abilities at the college-wide or university levels. Includes 
a bibliography of approximately 175 items. (DB) 
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EXECUTIVE SUMMARY 
THE DIFFERENTIAL COURSEWORK PATTESMS PROJECT 



AND THE DEVELOPMENT OF THE CLUSTER AMALYTIC MODEL 

The purpose of the "Differential Coursework Patterns (DCP) Project" was to 
determine the effect of different patterns of college coursevfork on the general 
learned abilities of students. To accooplish this end, a model for linking what 
coursework students took in college with what they learned in college was 
developed. The result was the Cluster Analytic Model. The Model groups courses 
appearing on student transcripts according to the distribution of assessment 
scores of those students. The Model uses precollege indicators of student 
learning to control for incoming student ability. It uses transcripts, rather 
than formal course or degree requirements as the representation of the college 
curriculum. The Model can use any number of assessment measures, including both 
quantitative and qualitative data. 

Over the past five years, the DCP Project team has developed and tested the 
Cluster Analytic Model in a variety of college and university settings. 
Consistently, the Model has identified and correctly classified approximately 8 
of every 10 courses undergraduates took according to the multiple measures of 
student learning used. To date, the Model has employed such criterion measures 
as the 9 item-types of the GRE General Test, Kolb's Learning Styles Inventory, a 
locally- developed instrument measuring the students' perception of the 
difficulty of the courses, and the ACT COMP examination subscores. Discriminant 
analyses of the course groupings according to these measures indicated an 
average correct classification rate of over 80 percent. 

Summary of the Cluster Analytic Model . The basic steps in the DCP cluster 
analytic model are summarized as follows. For simplicity of example, SAT scores 
are used as measures of incoming student ability. GRE item-type scores are used 
as measures of general learning. Coursework patterns are derived from the 
transcripts of graduating seniors at each of six postsecondary institutions. 
First, student achievement is determined by removing the predicted effect of SAT 
sub-scores on GRE item- type scores; the 9 item-type residuals serves as 9 
criterion variables of general learned ability during the student's 
baccalaureate years. Second, the mean residual scores of students enrolling in 
each course is determined and attributed to that course. Third, courses are 
taxonomized (or grouped into patterns) using one of two cluster analysis 
procedures according to the 9 GRE residual scores of the students. Fourth, the 
secondary validity of the clusters and the contribution of each of the 9 
item-type residuals to the clusters are established by discriminant analysis. 
Fifth, the resulting patterns are described in terms of the combinations and 
sequences in which the students enrolled in them and the predominant curriculum 
characteristics of the courses in the resulting pattern. Lastly, the coursework 
patterns are examined to determine the extent to which they are associated with 
a discipline or set of disciplines and the general education requirements of the 
college or university. 

Following these procedures produces a set of hypothesized relationships 
between coursework patterns and measures of general learned abilities. To 
provide primary validation of these relationships, a second sample of graduating 
seniors has been drawn from each participating institution, cmd the procedures 
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i^r Ihe cluster analytic model were repeated to determine the extent to which the 
coursework patterns and their relationship with the criterion measures of 
general learned ability were replicated. In one participating institution^ 
repeated samples have been drawn to determine enrollment trends and their effect 
over time. 

The conceptual framework for the cluster analytic model was derived from a 
review of selected literature. This review revealed that no one curricular 
model and no one analytical process clearly identified the effect of 
differential coursework on general learned abilities. Develoi^nt and testing 
of an analytic model for studying the differential effect of coursework in a 
complex curricular organization was required, A conceptual-^sipirical approach 
was adopted for the examination of the research problem, wherein a conceptual 
framework of student course decisions and selections guided the eis$>irical search 
for coursework patterns associated with gains in general learned abilities. 

Findings of the Research . The following is an excerpt of the ccxnbined 
findings and conclusions fron the first and second seniles of graduating seniors 
at one of the institutions participating in the DCP Project, Georgia State 
University, Both saiT^les were students who began their undergraduate education 
at Clayton State College (Horrow, Georgia) and then transferred to Georgia State 
to complete their baccalaureate program, graduating during the 1986'-87 and 
1987-88 academic years. The two university student sanples were similar to the 
population of graduating seniors for each year in terms of SAT scores, majors 
and demographic characteristics. The two san^^les were also similar to one 
another. There was no significant year-to-year variation in student 
characteristics, nor was there major variation in the characteristics of the 
samples relative to the population. 

There was major variation between the two san?>les in the courses they took 
and the general learning they exhibited. Twenty-five percent of the coursework 
in which 5 or more students from Sample #1 enrolled were not taken by Sairple #2 
students. While part of this variation is attributable to the system of general 
education at each institution, much is accounted for by the cumulative effect of 
the addition and deletion of courses frc»n the curriculums of these institutions 
as well. This process of curriculum revision at the department and program 
level accounts for a 5 percent per year change in course offerings. The 
cumulative effect of this change over the four years in which either sample was 
enrolled was over 20 percent. Prior research has characterized the curriculum 
as a static rather than a dynamic entity. This year-to-year variation places 
severe restrictions on generalizations about the effect of the curriculum on 
students. 

The growth in learning displayed in the test score results also varied 
between the prior two Clayton samples. Generally, Clayton students showed more 
gains in learning in Data Interpretation and Logical Reasoning. In the analysis 
of courses taken by 5 or more students, these two item^types explained large 
proportions of the score variance. In each of these analyses, from 4 to 6 of 
the GRE item^-types proved to explain most of the gains in student learning. 
Taking different coursework produces different effects in general learned 
abilities, and those effects varied among the students of the 1986-87 and 
1987-88 classes of graduating seniors who transferred from Clayton State to 
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Georgia State. "While there are significant student gains reported in this 
research, what they are varied significantly by year, by student ability group 
and by how the college emd university curriculums were defined. 

What does the Cluster Analytic Model tell us about assessment? The Cluster 
Analytic Model uses multiple loeasures of assessment. It provides colleges with 
information regarding the extent of variation in student assessi&ent results that 
is explained ^y any one of the measures used. This information can be helpful 
in a number of ways. Faculty and administrators need not decide on an ideal set 
of assessment measures. The extent to which such measiires may overlap in 
describing student learning can be identified. The mix of assessment measures 
appropriate to the goals of the college and the characteristics of the student 
population can be continuously monitored. When students show small amounts of 
growth on an indicator of student learning, either the college can develop 
strategies for improving student learning in the area identified, or discard the 
measure as inappropriate to the college and its students. The Cluster Analytic 
Model provides useful information to the college about the mix of assessment 
measures that reflects what the students learn and what the college intends to 
teach them. 

What does the Cluster Analytic Model tell us about the curriculum? The 
Cluster Analytic Model is a tool ideally suited to institutions of higher 
education with a distributional general education requirement «md a wide array 
of programs, elect ives and majors. For example, if one of the assessment 
measures a college selects is a test of analytic reasoning, then the Cluster 
Analytic Model can identify those groups of courses that students took who 
showed significant inrprovement in that area of general learning. Furthermore, 
the student population can be subdivided into high ability and low ability 
students, by gender, race or ethnicity, or by major. Then the Model can 
identify if the coursework associated with gains in learning among the total 
group is the same as that for the subgroups. Such appropriate information is 
valuable to curriculum planners. Courses in the general education sequence not 
found to be associated with gains in student learning can be revised, enhanced 
or dropped. Courses outside the general education requirements that contribute 
to gains in student learning can become candidates for inclusion in the general 
education curriculum. The extent to which general education courses affect the 
learning of both high ability and low ability students has relevance in deciding 
how wide ranging the distributional options should be or whether a core 
curriculum is appropriate for the students and the educational goals of the 
institution. 

How can the Cluster Analytic Model help with advising students? By linking 
the coursework students take with their iJi?>rovement in learning, the Cluster 
Analytic Model can be particularly valuable in advising students. First, it 
takes advising beyond the mere listing of formal degree requirements to the 
identification of those specific courses in which students of cca^rable 
interests, abilities and achievement have enrolled. Given several years of 
assessment data linked to the transcripts of graduating seniors, the Model can 
identify an array of courses taken by students who showed the largest gains in 
general learning in college. The Model is amenable to the development of a 
microcomputer-based advising system utilizing a relational database of prior 
students coursetaking patterns and assessment results. Such a con^uter-based 
advising system would yield an array of effective coursework tailored to the 
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abilities and interests of individual students and within the parameters of 
institutional degree requirements. Such a coiqputer-based advising system has 
been proposed as a logical extension of the DCP Project research. 
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INTRODUCTION 
The Differential Coursework Pat cerns Project 
and the Developiront of a Clu s ter Analytic Model 

The purpose of the "Differential Coursework Patterns" (DCP) Project is to 
determine the effects of different patterns of college coursework on the general 
learned abilities of undergraduate students. Student samples were drawn as 
volunteer graduating seniors from 5 geographically and curricularly diverse 
colleges and universities; each saaple represented the diversity of Scholastic 
Aptitude Test (SAT) scores and majors of the graduating class at each 
institution. The precollege general learned abilities of each sample of 
students was controlled in the research. The effect of SAT scores on the 
postcollege measures of student learning was partialled from the analysis; the 
remaining residual scores were used as criterion variables of student 
achievement. Thus^ exiting student general learned abilities were operationally 
defined by the residual differences from the predicted and observed scores on 9 
types of learning ("item- typos") measured by the General Test of the Graduate 
Record Examination (GRE). 

The GRE General Test consists of three sections: verbal, quantitative and 
analytical; within each test section are specific types of test items (i.e. 
Verbal: analogy items; Quantitative: quantitative coiqparisons; Analytic: 
logical reasoning). There is a total of 9 item-types within the General Test, 
and the 9 residual differences from the predicted and observed scores of 
students constituted the primary measures of exiting student achievement in 
general learning. Thus defined, exiting student achievement served as a metric 
in the analysis of the differential coursework represented in the transcripts of 
ecch institutional sample. 



Six institutions of higher education participated in the project. Each 
institution represented a different Carnegie classification. The institutions 
were: Clayton State College (Morrow, GA: a public cc^roraunity college at the time 
the enrollment of the saii?>le); Evergreen State College (Olynqpia, WA: public 
liberal arts college II); Georgia State University (Atlanta, GA: doctoral- 
granting university); Ithaca College (Ithaca, NY: con^^rehensive college); Mills 
College (Oakland, CA: private liberal arts I); Stanford University (Oakland, CA: 
private research university I)* 

The approach taken in the DCP project was eit5>irical. No a priori construct 
or model of curriculum was used to define what constitutes a differential 
coursework pattern. Rather, patterns were viewed as aggregations of (a) 
individual courses ^ (b) combinations of courses taken concurrently and (c) 
sequences of courses taken over time . The smallest unit of analysis for a 
coursework pattern was a single course. For the purposes of analysis, each 
course examined had 9 attributes represented by the 9 residual item-^type scores 
of students enrolling in the course. Courses with sufficient enrollment by the 
student sample were grouped according to the collective item-type scores of the 
students enrolling in the course; thus, each course examined had a mean residual 
gain score for each item-type. 

The effect of individual courses, coirfbinations of courses and sequences of 
courses on test score residuals was determined using hierarchical cluster 
analysis. The secondary validity of the coursework clusters (patterns) was 
derived and their relationsMp with the 9 item-'type residual scores was 
determined by discriminant analysis. The resultant patterns were then examined 
in terms of their role in the formal general education structure of the 
institution, the dominant type of instruction represented in specific patterns. 



and the nature of learning represented in those courses. 

The basic steps in the DCP cluster analytic model were as follows. 
Coursework patterriS were derived from thft transcripts of graduating seniors at 
each of six postsecondary institutions. First, student achievement was 
determined by removing the predicted effect of SAT sub-^scores on GRE item-*type 
scores; the 9 item-- type residuals served as 9 criterion variables of general 
learned ability during the student ^s baccalaureate years. Second, the mean 
residual scores of students enrolling in each course was determined and 
attributed to that course. Third, courses were taxoncsniEed (or grouped into 
patterns) using one of two cluster analysis procedures according to the 9 GRE 
residual scores of the students. Fourth, the secondary validity of the clusters 
and the contribution of each of the 9 item-type residuals to the clusters were 
established by discriminant analysis. Fifth, the resulting patterns were 
described in terms of (a) the combinations and sequences in which the students 
enrolled in them, (b) the predominant curriculum and instructional character- 
istics of the courses in the resulting pattern, and (c) the major, discipline 
and general education requirements represented in the courses of the pattern. 

Following these procedures produced a set of hypothesised relationships 
between coursework patterns and measures of general learned abilities. To 
provide primary validation of these relationships, a second 5an5>le of graduating 
seniors was drawn from each participating institution, and the procedxares of the 
clu&ter analytic model were repeated to determine the extent to which the 
coursework patterns and their relationship with the criterion measures of 
general learned ability were replicated. This report presents a further 
analysis of the combined two san?>les of these Georgia State University 
graduating seniors who began their undergraduate education at Clayton State 



College. 

The conceptual framework for the cluster analytic model wa's initially 
derived from a review of selected literature. This review revealed that no one 
curricular model and no one analytical process clearly identified the effect of 
differential coursework on general learned abilities. Development and testing 
of an analytic model for studying the differential effect of coursework in a 
con5)lex curricular organization was required. A conceptual-empirical approach 
was adopted for the examination of the research problem, wherein a conceptual 
framework of student course decisions and selections guided the eirpirical search 
for coursework patterns associated with gains in general learned abilities. 

During the first project year (1986-87), a model was postulated to 
determine effects associated with differential coursework patterns on selected 
measures of general learned abilities of students. Requirements of this model 
were that it should have explanatory power regardless of institutional setting 
and should function independently of the particular assessment tool (i.e., the 
GRE exams) chosen. The generalizability of the model to other* institutional 
settings was tested in the second year of the project. The model as initially 
postulated was refined using a supplemental historical data set of student 
transcripts and test scores from Georgia State University (GSU) . The 
preliminary testing and analysis of the model occurred later in the first 
project year, when the first san^le of graduating seniors at GSU(San^le Group 
#1) was given the GRE and LSI instruments. Results from that administration 
were analyzed in previous DCP Project reports (Ratcliff, 1988, 1989). 

During February 1988, a random san^sle of graduating seniors at the 
institutions participating in the study (Evergreen State, Ithaca, Georgia State, 
Mills and Stanford) were given the GRE and LSI. Sample Group »2 at GSU was 
given the GRE, LSI and MPDQ in February 1989. Educational Testing Service 



scored the GRE tests, prepared a data tape of score results, and retired and 
released the test forms to the five institutions tested in May 1988 for San5>le 
II and in May 1989 for San5>le #2. This report presents the findings of the 
application of the cluster analytic model to the tests and transcripts of a 
coaibined sainple of 1987-88 and 1988-89 graduating seniors at Clayton State 
College. 

Organization of the Report 
This report is divided into 8 sections. Each section describes a major 
component of the research activity. Section 1 describes the theoretical basis 
for the differential coursework hypothesii, the development of the cluster 
analytic model as a means of testing it and the research objectives of the DCP 
project. Section 2 provides a detailed description of the procedures and 
n«thodology used in conducting the research. The cluster analytic model is 
described procedurally. Section 3 portrays the major characteristics of the 
graduating seniors whose transcripts and test scores were examined as part of 
the data gathering and analysis. Section 4 describes the characteristics of the 
courses found on the students transcripts in terms of the criterion variables: 
the GRE item-type residuals. Section 5 depicts the intended curriculum of the 
institution, its goals, curriculum organization and structure. Courses found on 
the students transcripts are con^^ared with the general education requirements of 
the institution. Section 6 reports the findings from the combined san^le using 
the quantitative cluster analytic procedures described in Section 2. Section 7 
examines the comparability of coursework between the Clayton State transfer 
students and the Georgia State native students. Each institution had a 
comparable general education requirement and courses at each institution were 
subject to the same course numbering scheme used throughout the University of 
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Georiga System, section 8 offers a sumroary, conclusions and reconiinendations 
emanating from the research. 
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1 . Vfriat Are "General Learned Abilities" 
The ::lu5ter analytic nvodel does not present nor does it rely on a 
particular theory of student learning. There is widespread disagreement on what 
constitutes "general learned abilities", and that disagreement is manifest in 
the variety of general education goals and degree requirements found in American 
higher education (Bergquist, Gould & Greenberg, 1981; Carnegie Foundation for 
the Advancement of Teaching, 1979; Gaff, 1983; Levine, 1978). Vlithin the term 
"general learned abilities", we mean to include such frequently used terms as 
"higher order intellectual processes" (Pascarella, 1985), "academic 
competencies" (Warren, 1978), "generic coif5>etencies" (Ewens, 1979), "generic 
cognitive capabilities" (Woditsch, 1977), and "general academic ability" 
(Conrad, Trisman & Miller, 1977). Disagreement on terminology is but one aspect 
of the problems associated with measuring the general learning of students as 
undergraduates . 

Current notions of how to assess coL.sge outcomes call for multiple mea- 
sures of student achievement. No one measure has been found to accurately 
reflect the variety of definitions of general learning and cognitive develop- 
ment, the mixture of curricular goals and institutional characteristics found 
across the landscape of higher education and among the diversity of instruction- 
al procedures and curricular organizations of undergraduate higher education. 
The result has been a call for multiple measures of assessment of student learn- 
ing. Policymakers and academic leaders tend to believe that since colleges and 
universities have broad missions and goals, assessments should be con^arably 
broad enough to provide evaluation about as many institutional intents as 
possible (Loacker, Cromwell & O'Brien, 1986; Nettles, 1987). 

Given the variety of terms, intents and theoretical frameworks used to 
explain "general learned abilities", the research design presented here is 
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criterion-referencea . That is, the design is not based on any one notion of 
what constitutes "general learned abilities" or any one college curricul.ir 
structure intended to prcanote student cognitive development. The design permits 
the use of multiple and different measures of student outcomes, although the 
design is being developed and tested using one set of such measures. The 
validity of the assessment of student learning within this research design, 
then, is dependent upon the validity of the outccsrie measures selected, rather 
than the degree to which the student outcomes suffice a global ( goal-free ) 
measure of student learning. The design fulfills the need for multiple measures 
and criterion-referenced measures of student learning. However, because the 
design described below is not dependent upon any given college curricular 
structure or organization and, in fact, will be tested in five very different 
higher education institutions, it is free of bias engendered by specific 
institutio.ial goals. 



What constitutes student achievement? 

Another question encountered in determining what constitutes general 
learned abilities is that of what composes the gains resulting from a college 
education. Simply measuring how graduating seniors perform on a series of tests 
is not a sufficient basis for generalizations about the effect of college on 
student achievement. First, the assessment of student outcomes is heavily 
affected by the students' academic achievement prior to entering college (Astin, 
1970a, 1970b; Bowen, 1977; Nickens, 1970). In fact, standardized tests used for 
college admission, such as the Scholastic latitude Test (SAT), have been shown 
to be strongly correlated with tests used for graduate and professional school 
admissions, such as the General Tests of the Graduate Record Examination (GRE). 
These correlations have been demonstrated for the total and sub-scores on the 
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two tests, suggesting that a large proportion of what postcollege tests, such as 
the GRE, ineasure are attributable to student learning prior to college. Since 
the SAT and GRE examinations were used in the development of the DCP cluster 
analytic model, further examination of their capacity to measure general learned 
abilities follows. 

Nichols (1964) studied the effects of different colleges on student ability 
as measured by the Graduate Record Examination's Aptitude Test. A sample of 356 
National Merit finalists attending 91 colleges was used. The effects of college 
characteristics, major fields and types of colleges on student GRE scores were 
examined while controlling f c • the precollege student characteristics. Strong 
correlations existed between the students' SAT scores and the students' GRE 
scores which ranged frcsn .65 for the verbal to .76 for the Quantitative. The 
results from this research indicated that the college students attended did have 
an effect on their GRE performance. There was a significant tendency for 
colleges to separate Verbal and Quantitative scores and raise one while lowering 
the other. Therefore, "the effect of college appears to be one of directing the 
students' abilities into verbal or quantitative channels rather than affecting 
the overall level of ability" (p. 52). 

Rock, Centra and Linn (1970) and Rock, Baird and Linn (1972) examined SAT 
and GRE area test scores of 6,855 students who graduated from ninety-five col- 
leges, predominantly small, private liberal arts institutions. The correlation 
between college means on SAT~Verbal and GRE-Total was 0.91. The fact that the 
standardized precollege and postcollege tests, such as the SAT and GRE, are 
strongly correlated should not be sur^jrisirg. Students typically bring 12 or 
more years of formal education with them upon entrance to college, and since the 
college years traditionally constitute 4 or 5 years, a large proportion of 
general learned abilities of students should be attributable to learning 



experiences prior to college admission. 

The strong relationship between the SAT and the GRE is both an asset and a 
liability. The use of the SAT subscores as precollege measures and the GRE 
item-types as postcollege measures does provide a basis for controlling the 
effects of student academic achievement using cou^jarable definitions of general 
learned abilities and con^sarable testing procedures. However, the strong corre- 
lation between the two tests leaves only a small amount of explained variance 
between precollege and postcollege scores to attribute to general learning 
associated with a baccalaureate program. 

The dileirana posed by the use of the SAT and GRE as measures of general 
learned abilities among students is exacerbated by the student population dif- 
ferences upon which the tests are normed. Adelman (1985) estimated that 25%-30% 
of graduating seniors take the GRE General examinations, while 60% of the grad- 
uating high school students take the ACTs or SATs. These rough percentages are 
understandable since fewer individuals choose to continue their education from 
bachelor's to graduate study than do those who choose to go to college from hiyh 
school. Nevertheless, a consequence is that the GREs are normed on a highe : 
ability population than the SATs (Pascarella, 1985). The individuals taking the 
GREs constitute a self -selected sair^Jle, driven in part by the requirements of 
graduate schools, professional schools, departments offering graduate degrees, 
and organizations requiring such examinations as part of the formal application 
for fellowships and scholarships (Adelman, 1985). In the DCP Project, a 
concerted effort was made to have each sample of graduating seniors not limited 
to this self-selected group who would have t£iken the GRE as a normal event 
toward their application to graduate school. DCP Project sannpling procedures is 
discussed in Section 3. Nevertheless, it is i-nportant to note that the Graduate 
Record Examination can be accurately viewed as a measure of a student's 



- 4 - 



ERIC 



predicted performance in graduate school as well as a measure of that student's 
general academic accomplishments as an undergraduate. 

The GR£ and SAT tests have been criticized for a) the bias resulting from 
groups upon wh-^ch they were nonned (Adelman, 1985; Nettles, Thoeny & Gosman, 
1986) and b) their limitation in iroasuring higher order reasoning skills. These 
criticisms notwithstanding, the GRE and SAT tests do provide an econcsnical, 
practical, and valid way of measuring selected general learned abilities while 
controlling for the incoming academic accCT?>lishiDents of freshmen (Astin, 1968; 
Hendel, 1977). Critics of the GRE and SAT as measures of general learned 
abilities attack the validity of the measures themselves. These criticisms 
primarily are based on the use of sub-^scores and total scores of the tests. The 
use of either the GRE or SAT item-type scores as multiple measures of general 
^earning has previously not been widely explored (Adelman, 19 j). The DCP 
Project provided a detailed assessment of GRE item-types as discrete measures of 
general learned abilities. 

While the GRE and SAT tests were used in the development of the DCP cluster 
analytic model, it is not dependent upon these sets of measures. The model can 
be en^loyed using another set of correlated precollege and pcstcollege measures 
in a longitudinal analysis (for exanqple see Pike, 1988.) For pragmatic and 
economic reasons, the cluster analytic model was developed using SAT and GRE 
tests. The model was initially developed using the two sub'-tests of the SAT and 
GRE: the SAT Verbal Test (SAT-V), the SAT Mathematical Test (SAT-M), the GRE 
Verbal (GRE-V), and the GRE Quantitative (GRE-Q). Subsequent development and 
testing of the model enqployed the 9 item-type parts of the GRE as multiple mea- 
sures of student learning. Therefore, the cluster analytic model will be des- 
cribed in terms of 9 measures of student general learned ability. 



Prior research suggested that the 9 item-type scores were independent 
measures of general learning. Wilson (1985) examined the criterion-validity of 
the 9 item- type part scores of the GRE General Test to the prediction ct 
self -reported undergraduate grado point average (GPA>. For his research, Wilson 
used the GRE scores of 9,375 examinees in 9 different fields of study 
representing 437 undergraduate departments from 149 colleges and universities. 
Data were first standardized within each department, then pooled for analysis by 
field of study. Results suggested that the GRE item- type scores did 
differentiate undergraduate GPA by field of study. This research and other 
studies (Powers, Swinton & Carlson, 1977; Swinton & Powers, 1982; Wilson, 1974) 
indicated that the 9 item-type subparts of the GRE measure different and 
somewhat unique general learned abilities. Initial testing of the DCP model at 
four colleges and universities also indicated that the item- types constituted 
independent measures of general learned abilities (Ratcliff, 1988, 1989). 

The GRE General Test consists of three sections: verbal, quantitative and 
analytical; within each test section are specific types of test i cems (i.e. 
Verbal: analogy item; Quantitative: quantitative con^arisons; Analytic: logical 
reasoning). There are 9 item-types within the General Test; the residual 
differences between the observed GRE scores (postcollege measure) and the GRE 
scores predicted by the students' corresponding SAT scores (precollege measure) 
were used to gauge general learned abilities attributable to the students' 
undergraduate education. The residual differences between the observed and 
predicted values of each GRE item-type served as the 9 measures of student gains 
in general learned abilities during the time in which the student was enrolled 
in college. Thus in an economical and practical way, these item- type residuals 
represented a set of multiple measures of general learned abilities which 
accounted for and controlled the effect of precollege student achievement on 



postcollege student outcomes. 

What constitutes a coursework pattern? 

The prevalent way to view the collecje curriculum is by its intentions, 
rather than by its results (Warren, 1975). Since measuring the effects of the 
curriculum is problematic, it is not surprising that many studies presume rather 
than test the effect of different patterns of coursework. 

The college curriculum is substantiative, additive and temporal. In terms 
of cognitive theories of curriculum development, both content and process con- 
tribute to developmental learning in students (Tyler, 1950; Taba, 1962). Essen- 
tialist and constructionist theories of curriculum stress combinations of sub- 
jects (core curricula, great books, etc.) as influential on general learned 
abilities of college students (Fvihrmann and Grasha, 1983). The medieval univer- 
sity curriculum was organized according to combinations and sequences of courses 
as well as individual subjects (Rudolph, 1977); the seven liberal arts were 
sequenced into the prerequisite subjects of the guadrivium (arithmetic, geo- 
metry, astronomy, anJ music) and the higher ordtr subjects, the trivium (logic, 
grammar, and rhetoric). Together, the quadrivium and trivium provided an indi- 
vidual with the general learned abilities needed to study the three philosophies 
of Aristotle: natural philosophy (physics), n>oral philosophy (ethics), and men- 
tal philosophy (metaphysics). These combinations and sequences of coursework 
have been generalized more recently into concepts of breadth and depth as 
criteria by which to describe higher education curricula (Blackburn et al., 
1976). 

While the notion that combinations c'i concurrent coursework and devel- 
opmental sequences of coursework lead to effects in the general learned abili- 
ties of students is derived from the medieval university, it is underscored and 



further supported by the research of conteii5>orary developmental theorists. 
Perry (1968) for example, stated that development "consists of an orderly pro- 
gression of cognition in which more complex forms are created by the differenti- 
ation and reintegration of earlier, sii!?>ler forms" {p. 44). 

The value of curricuiar substance and sequence are presumed in fonmilations 
of core curricula, in the four levels of study (freshman, sophomore, junior and 
senior years), in the corresponding practice of assigning course numbers accord- 
ing to those divisions, and in the practice of assigning course prerequisites- 
To assess the impact of these coursework patterns on the general learned abili- 
ties of students, the additive, substantiative and sequential characteristics of 
student course- taking need to be examined. These notions of what ought to be 
taught and what students ought to learn presumably represent the philosophical 
and educational aims of the particular college. 

Nevertheless, a distinction should be made between those patterns of 
coursework intended to fulfill undergraduate program and degree requirements and 
those patterns of coursework which students actually choose (Boyer & Ahlgren, 
1981, 1982, 1987; Warren, 1975). Intentional patterns of coursework are 
provided in a variety of publications issued by the institution: the college 
catalog, the annual schedule of times and days of courses, and program descrip- 
tions issued by departments and divisions within the college. Richardson et al, 
(1982) provide evidence that a minority of students may consult these statements 
of curricuiar intent prior to making decisions about which courses to choose. 
Other forms of intentional coursework patterns are the lists of courses or sub-- 
jects required for certification or licensure in a particular profession, occu- 
pation or technical field. Such lists of coursework may be con^iled by practi- 
tioners and academics of a given discipline or profession to accredit college or 
university programs. Just as the curriculum of a particular college may repre- 



sent the philosophy and educational aims of that institution, so too may the 
certification, licensure and accrediting standards articulate the intentions of 
state, reg-.onal, disciplinary and progrananatic associations. All are intended 
patterns of coursework in the curriculum whose measure of effectiveness, in 
part, is the extent to .^hich these patterns acconplish their aims in practice. 

In a college curriculum, a single course may be the smallest unit of 
analysis. In assessing the ingjact of the curriculum on the general cognitive 
development of students, the course constitutes a datum in the analysis. A 
pattern of data is a design resulting from "the relation among a set of objects" 
(Romesburg, 1984, p. 278). In this case, the objects are courses. Therefore, a 
coursework pattern is a design resulting from relationships among courses. A 
cluster of courses (objects) is a set of one or more objects found to be similar 
to one another according to a given set of attributes. In the DCP Project, 
courses were grouped according to the extent of gains (or losses) in general 
learning of the students en*.-olled in t:, courses. Thus, for the purposes of the 
DCP cluster analytic model, a cluster of courses is used to denote a pattern of 
coursework with an empirically derived set of relationships. Stated another 
way, a cluster of courses is a pattern based on the actual enrollment of a 
cohort of students, rather than the intended enrollment patte'.-n of the college 
or university or the set of courses any one student selected. This distinction 
is important in order to differentiate between the consequences of the college 
curriculum and its intents. 



Sources of data for coursework patterns 

Argximents about what is and what is not an effective college curriculum are 
for the most part based on seasoned speculation, nostalgia about academic tradi- 
tions, and unrealistic expectations of curriculeu: coherence among and within the 
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over 3,000 colleges and universities in the United States (Conrad, 1986). In 
most instances, the data used in describing the status of general education are 
derived from catalog studies and enrollment analyses. These data may not pre- 
sent an accurate picture of general education as it functions in students' 
programs . 

Catalog data provide evidence of trends in offerings but ignore student 
behavior. Catalogs indicate which courses a student should take to fulfill 
degree requirements and describe the intended contents or outcones of the 
courses; collectively, this cotg^^ndium of courses and articulation of degree and 
program requirements ideally leads to the accoirplishment of the educational 
philosophy and curricular goals of the institution. White (1979) claijned that 
"college catalogs generally have little to do with reality*', and that "the 
educational ideas expressed are subsequently neither perceived nor accom- 
plished" (p* 39). The ideals expressed are important, ''yet when subjected to 
the rigorous scrutiny of time and experience, the academic promise is often not 
realized" (p. 39). White sugge^Led that all collegiate programs, including 
general education, should be assessed in light of their original claims and 
promises. 

Dressel and DeLisle (1969) investigated college curriculum trends using 

catalogs as priinary sources of information. They conceded that data derived 

from studies like theirs must be interpreted circumspectly, asserting that: 

, . . ambiguities and contradictions arise in the use of catalogs for 
research of curriculum practices, because what appears in the catalog 
as policy is in reality often left to interpretation of individual 
advisors and individual departments, and what is in reality required 
by an individual department is often not stated as policy in the 
catalog (p. 75). 

. . . the limits described by departments tend to be more exacting and 
demanding than those stated in the institutional requirements listed 
by the college. Thus, a question arises as to whether each student 
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actually has flexibility and innovation claimea in the general state- 
ments or whether the department control of the major serves a 
limiting and inhibiting factor in this respect (p. 78). 

Dressel and DeLisle also noted that catalogs, as sources of information about 
the curriculum, cannot be examined to determine the extent to which curriculum 
policies and statements correspond to actual course-taking practices of stu- 
dents. Because of ambiguities, inaccuracies, discrepancies, and emissions in 
wording in catalogs, the interpretation of the catalog varies significantly 
among both students and advisors. Also, catalogs often do not present a ration- 
ale for course requirements, nor is there a way to determine how, why, or when 
requirements were introduced. Likewise, there is no way of determining "whether 
claimed articulation of liberal with professional education and of breadth with 
depth has been successfully achieved" (p. 79). In sum, catalogs do not provide 
information about the consequences of the coursework on student learning, and 
therefore, are not appropriate for learning outcomes research. 

Assessments of the outcomes of college have little meaning unless compared 
with either a normative group of students or the intended general educational 
goals of the institutions. Catalogs are often the roost con^rehensive statements 
of the intended substance and sequence of intended learning activities. From 
this standpoint, they provide a basis for ccaparison of college intentions with 
college outcomes. The DCP Project closely examined the extent to which the 
college catalog represented the formal curriculum students experience in 
college. This comparison is presented in Section 5 of this report. 

VJhile college catalogs are inadequate in describing course- taking behavior 
of students, enrollment figures as data sources also provide limited inform- 
ation. Although such figures are often used for evidence of curricular trends 
among undergraduates, enrollment analyses do not describe the actual course pat- 
terns of enrollees; however, they do reveal the extent to which different seg- 
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loents of the college population choose particular majors or general education 
courses (intended coursevrork patterns). Noting the inability of enrollment 
analysis to determine student reasons for course selection, Friedlander (1979) 
used transcript analysis to examine the effect of changes in the con^josition of 
the comnunliy college student body on humanities enrollments. Adelmaii, in an 
analysis of the Postsecondary Education Transcript Sample (PETS) of the National 
Longitudinal Study of the High School Class of 1972, suninarized the advantages 
of transcripts as unobtrusive, empirical artifacts of student learning, 
"transcripts neither exaggerate nor forget" (l°d9, p. 1). 

Student transcripts as a data source 

Student transcripts are a rich, unobtrusive and problematic source of 
information about student course- taking behavior. Warren (1975) used 
transcripts to determine coursework patterns among college students in a study 
of 50 history graduates of different four-year colleges. The student 
course-selection patterns in history, as revealed in these transcripts, 
indicated that within the discipline there were at least three or four different 
history programs. This finding demonstrated that although students receive 
similar degrees, they do not necessarily have the same educational experiences. 
Warren's study suggested that students shape their own curricula as they 
exercise options in choosing courses to con?>lete credit hour requirements. 
Furthemxjre, Warren demonstrated that transcripts could be used to discern broad 
curricular patterns. 

Blackburn and associates (1976) used transcript analysis in their investi- 
gation of curricular change and course-taking behavior in two and four year 
colleges and universities between 1967 and 1974. One of the goals of the study 
was "to determine how students utilize elective time" (p. 20). Since some 
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two-year transcripts did not indicate institutional requirements for both 
general education and the major ^ the researchers could not ascertain what 
courses were elective and which were prescribed. Consequently, the transcripts 
from two-year colleges were eliminated from the sanr^le used to describe student 
course- taking patterns. The question of treatront of transfer students in 
transcript studies and college impact studies has also been a consistent 
methodological problem (Astin, 1970a). 

?rather and associates (1976) used transcript analysis to study under- 
graduate grading practices at Georgia State University and investigated 
differences in grading patterns by major fields of study while controlling for 
such antecedents as scholastic aptitude, demographic background, course types, 
and longitudinal trends. They found that major field was strongly associated 
with the grades students received in courses throughout the curriculum. This 
research and previous grade studies supported the proposition that the various 
parts of the curriculum have different grading standards, arguing against the 
use of GPA as a proxy measure of general learned abilities in college students. 
The studies by Warren (1975) and Blackburn and associates (1976) used small 
samples of transcripts (5 percent) due to the time demanded in reading and 
assessing all the courses on each transcript. Prather and associates, however, 
used an electronic database of student transcript information to examine an 
institutional cohort; use of electronic databases of records enabled the 
researchers to examine larger samples of student records, thereby permitting 
analysis of larger sections of the curriculum. 

Transcript analysis has been used to examine the general education 
component of the undergraduate curriculum as well. The dean of instruction and 
curriculum planning at the University of Pennsylvania used transcript analysis 
in an effort to determine which courses among the many listed in the college 
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catalog were actually selected by arts and sciences graduates (Carnegie 
Foundation, 1979). He found that 1976 graduates of arts and science programs 
had selected "a core of 29 courses" (p. 97) in the curriculum. However, not 
all students chose the same combination of courses, and "many of the thousands 
of courses in the catalog that were not included in the core list were found on 
individual transcripts" (p. 97). This study illustrated one of the persistent 
problems in using transcript analysis to identify course-taking patterns: the 
enormous range of possibilities of course sequences generated by student choice 
in a large, multi-purpose university. It also suggested that, for whatever 
reason, there is a limited number of courses which most students select to 
complete the general education requirements of the undergraduate program. 

Beeken (1982) used transcript analysis to examine the course-taking 
behavior of a sample of students in three Virginia conaminity colleges. The 
purpose of the study was to determine the number and types of general education 
courses selected by students to meet the general education requirements of the 
Virginia Community College System. The study did not confirm the conclusion of 
the Carnegie Commission that the general education curriculum was a "disaster 
area", although the programs of many students did not present a balance of 
disciplines; students apparently minimized the number of mathematics and science 
courses in their program of study. Both those who completed an associate of 
arts degree and those who did not exceeded the minimum requirements for general 
education courses. The number of courses taken in different curricular areas of 
general education were related to enrollment status, age, and sex. 

One of the largest collections of student transcripts is the Postsecondary 
Education Transcript San^sle (PETS) on the National Longitudinal Study of the 
High School Class on 1972 (NLS). PETS data consisted of 22,600 students 
transcripts. Vfhile NLS has several precollege measures of achievement (high 
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school grades^ SAT, etc.) and the coursework selected by the students who 
attended college is represented, the NLS data has no available 
post-baccalaureate measure of general learning. Adelinan (1989) used t^LS/PETS 
and the NLS 5th Follow-Up Survey to deinonstrate relationships between coursework 
taken in conrounity colleges and success in attaining bachelors and advanced 
degrees, career aspirations and plans, and self ^reported attributes of the jobs 
the students held 15 years after high school graduation. In this analysis, 
transcripts proved to be a powerful, non-obtrusive measure of the relationship 
between what a student planned, what they studied at college, and what the 
nature of their work was a decade and a half later. 

In a limited number of studies, transcripts have been found to be a useful, 
valid and reliable source of information on student course-staking behavior. 
They provide evidence of the combination, sequence and performance of students 
in the patterns of courses in which they enroll. As archival records^ 
transcripts are unobtrusive data. Vlhile most studies have limited their use of 
transcripts to a manual examination of a small san5>le of student records, there 
is evidence that such records, stored on a college or university coirputer, can 
be used to examine the course-taking behavior of a whole class, cohort or 
population of students. Thp cluster analytic model for determining the 
associated effects of coursework patterns on the general learned abilities of 
college students uses transcripts in precisely this manner. Transcripts 
maintained on an electronic database can be merged with student score residuals 
for the purposes of assessing the effects of the curriculum on student 
learning. 
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A conceptual framework for analyzing coursework patterns 

The differential coursework hypothesis posits that students who enroll in 
different coursework will show different levels or types of gain in general 
learned abilities (Benbow and Stanley^ 1980, 1982, 1983; Pallas and Alexander, 
1983). While all the courses Student X chose collectively affect X's gains in 
general learning, the effects of an individual course on X's transcript may vary 
in its contribution to such an effect. The effect of individual courses may be 
mediated by prior student aptitude, ability, achievement and interests. 

For the purpose of analysis, thu effects associated with a particular 
course are proxied by the residual scores of the students who enrolled in that 
course. A pattern of data is a design resulting from "the relation among a set 
of objects" (Rcanesburg, 1984, p. 278). In this case, the objects are courses. 
Therefore, a coursework pattern is defined here as a set of courses having 
comparable effect on one or more student residual scores. The differential 
coursework hypothesis is rejected when no patterns are discernible among the 
data--when the residual scores of students are uniformly attributable to 
enrollment in any and all courses in the curriculum. The hypothesis is affirmed 
when students who perform well on one or more postcollege measures tend to 
enroll in certain courses and not others. 

At this point in the inquiry, the DCP cluster analytic model is not 
concerned with reasons for the effect. Rather, the next step is to iaentify and 
classify courses according to the score gains of students who enrolled in them, 
regardless of the factors that may have brought the students to enroll in the 
courses. Also, the model is not yet concerned with characteristics of the 
students, although those characteristics may covary with the course selection or 
achievement variables (Elton and Rose, 1967; Prather et al., 1976). Thus, in 
the examination of the effect of course patterns, there is no implication of 
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direct causality of course patterns upon achievement (Astin, 1970a, 1970b). 

There is reason to presume that the effect of a single course may vary 
according to what place it holds in the pattern of courses a student chooses 
(Prather et al., 1976). For exanqple, if courses at a particular college are 
sequenced according to level (e.g,^ 100 level courses are intended for freshmen 
and 400 level courses are intended primarily for seniors), the effect of History 
101^ •^Survey of Western Civilization", may differ for Student X who enrolls as a 
first term freshmen from Student Y who enrolls as a final term senior. 
Conversely, logic holds that the effect of History 451, "20th Century American 
Foreign Policy", may differ for the first term freshman and the last term senior 
(Rudolph, 1977; Veysey^ 1973). If a course is viewed as contributing to the 
residual score for a particular measure of general learning, then a coursers 
effect may vary according to itF place in the student's pattern of courses • 
Therefore, course sequencing should be considered in the examination of course 
patterns {Bergquist et al,, 1981), 

Likewise, the effect of a particular course may be associated with the 
effect of other ccurses in which the student may be concurrently enrolled. 
Richardson et al. (1982) and Roueche and Snow (1977) noted that students may be 
advised to enroll in elementary writing or mathematics courses concurrently with 
other courses requiring the basic skills these elementary courses teach. Under 
such practices, the student may have much less chance to succeed in college. 
Traditionally, the combination of courses in which a student enrolls within a 
given term is presumed to have effect (Bergquist, et al, 1981; Rudolph, 1977; 
Veysey, 1973) and therefore also should be considered in the analysis of course 
patterns. 

Thus, there is a great deal of uncertainty associated with why a particular 
student chooses a particular course at a particular time in his/her program of 
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study. A poor grade in "Trigoncmjetry" may cause Student Y to select a remedial 
mathematics course over "Introduction to Calculus'^ Student who received a 
high grade in "Trigonometry", may not enroll the following term in "Introduction 
to Calculus" because the time it is offered conflicts with a course Student X is 
required to take within his/her major. Or the Calculus course may be filled 
when Student X tries to enroll. Many factors shape the cocibinatlon of courses a 
student chooses in a given term and the sequence of courses represented across 
terms in the transcript. 

A modern research university may present 2,500 to 5,000 undergraduate 
courses from which a student may choose 35 to 45 courses to con^lete the 
baccalaureate degree. Each semester or quarter a student enrolls, that student 
selects several courses. Each term of registration represents a stage in the 
overall decision-making process which generates the patterns of coursework found 
on the student transcripts at the time of graduation. Each enrollment decision 
is limited and shaped by those courses in which the student has previously 
enrolled and the various degree requirements and prerequisites that are enforced 
during the registration process. At each successive decision-point, the student 
is progressively more immersed in the college environment, the norms and values 
of the student's peers, and the norms, values and expectations of the subjects 
the student selects to study. 

The analysis of the pattern of courses a student chooses is a sequential 
decision-making process wherein certain conditions exist: 

!• students make course selections in an environment of uncertainty 
about the consequences of their choices; 

2. there are multiple reasons why students enroll in each course; 

3. there are multiple options available to the student at each 
decision-point (term registration period); 
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4. student course selections are sequential; there are different 
decision-points (terms) in which parts of the coursewcrk pattern 
are chosen^ with prior decisions having soine bearing on future 
decisions. 

Under the conditions listed above, students may choose courses to minimize 
uncertainty and risk (i.e., seek what they perceive to be '^easy^* courses) » They 
may also seek courses which will maximize the efficiency (i.e«, fulfill degree 
and graduation requirements with a minimum amount of time), or maximiEe 
effectiveness (e.g., "it's a hard course, but I need to pass it if l*m going to 
major in engineering"). In this way, the succession of registration decisions 
comprising the student's pattern of coursework concept\;ally represents a 
multiple-stage decision-making process (Buchanan, 1982; Bunn, 1984). 

According to Pace (1979), one variable in student develojmient is the amount 
of time and effort invested by the student. This premise, that student involve-- 
ment in learning advances student achievement, guided the recomroendations of the 
NIE Study Group's Report on Conditions of Excellence in American Higher 
Education (1984). Not only the kind and quality of cognitive activities in 
which the student engages, but also the level of effort exerted by the student 
in understanding and using the knowledge and abilities gained influence the 
quality of student learning. The student's effort in courses is "impressed" 
(Pace, 1979) by attitudes of the perceived usefulness of the course and the 
perceived difficulty of the course. These perceptions influence the kind and 
quality of student investment in learning. Coyne and Lazarus (1980) found that 
such investment involved both cognitive and subjective elements, leading to 
whether the experience is viewed as a challenge or a threat. The perceived 
difficulty of courses influences student enrollment decisions and thereby 
contributes to the multiple-stage enrollment decision-^making process through 
which the student compiles his or her particular collection of coursework. 
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In sunsnary^ the literature suggests 9 number of possible interactions 
between student and curriculum each time a student makes course selections. The 
effect of courses on general learned abilities may vary according to the course 
itself^ the tiine of enrollinent in the student's baccalaureate program, the 
concurrent or sequential relationship to other courses in which the student 
enrolls^ the predcsninant learning style of the course and of the student, the 
curricular design of the course, and the risk-taking behavior the student 
exhibits at each enrollment decision-point. The DCP cluster analytic model 
calls first for the identification of student achievement (i.e., student score 
residuals), second for the classification of courses found on student 
transcripts into patterns according to their associated effects on the student 
score residual, and thirdly, for the further classification of courses within 
each identified pattern according to a) sequence and combination, b) the 
learning styles of the students, c) the risk-taking behavior of the students, 
represented by their perception of the difficulty of selected coiurses in which 
they enrolled, and d) the common curricular characteristics of courses found 
within a given pattern of coursework. The model provides a basis for examining 
the extent to which the empirically-derived patterns of coursework reflect 
institutional mission and curricular goals, general educational requirements, 
the values, norms and mode of inquiry represented by the disciplines studied, 
and the demographic characteristics of the students • The model acccsiq^lishes 
these objectives through the use of cluster analysis, a statistical procedure 
which has been used throughout the physical and social sciences to derive 
empirical taxonomies of objects in a variety of settings. Cluster analysis, 
since it has been infrequently en^loyed in education, is described in greater 
detail in Section 2. 
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Research objectives of the PCP Project 
The objectives of the DCP Project: 

1. To determine student academic achievement in general learned 
abilities gained during the baccalaureate program. This 
achievement was measured by the fixed criteria of the residual 
scores on the 9 item-type subparts of the General Test of the 
Graduate Record Examination (GRE)^ once the effects of precollege 
achievement as measured by Scholastic Aptitude Test (SAT) scores 
were removed. 

2* To classify the coursework taken during a student's baccalaureate 
program according to its associated effects on the student's 
general learned abilities^ as measured by the GRE* This coursework 
was determined by a cluster analysis of student transcripts wherein 
courses will be described and classified into patterns according to 
the GRE residuals of the students enrolling in them. 

3, To test the secondary validity of the coursework clusters and to 
identify outlying cases within each cluster of courses, 
discriminant analysis was applied to the results of the cluster 
analysis. Through examination of pooled within-group correlations 
of discriminant functions with GRE item-'types and the cluster group 
means on each discriminant function, relationships between cluster 
groups and item- type resic aal scores was determined. 

4. To describe the resulting patterns of coursework according to: 
sequences and combinations of courses within the cluster, according to 
term of enrollment data found on student transcripts and the common 
curricular characteristics of the institution. 



Samples used in the initial development 
of the cluster analytic model 



For the purposes of building and testing the cluster analytic model, an 
historical database was developed at Georgia State University. This database 
consisted of 1,024 students who began their baccalaureate education at GSU, 
graduated, and then continued in graduate or professional education at GSU. To 
qualify for inclusion in this database, a student must have con^^leted 14 or more 
quarter credits at GSU. This same criteria also applies for Clayton State 
students. A student who completed more than one quarter (15 credits) at another 
institution prior to enrolling at GSU was not included in the Historical Group. 
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The database was drawn from all student transcripts at GSU between 1975 and 
1985. This population was selected because (1) it was a readily available 
database, (2) prior research demonstrated that it was amenable to statistical 
analysis (Prather & Smith, 1976a, 1976b; Prather, Smith & Wadly, 1976), and (3) 
the transcript records contained SAT and GR£ information as well as courses 
taken* 

From this database, 56 student records were found to contain the Scholastic 
Aptitude Test verbal scores (SAT'-V), Scholastic Aptitude Test mathematical 
scores (SAT-N), Graduate Record Examination verbal score (GKE-V) and Graduate 
Record Examination quantitative score (GRE-^Q). All student identification 
information was removed from this database at GSU, so the individual identity of 
the student was unknown to the researchers developing the cluster analytic 
model. It should be noted that these 56 student transcripts were representative 
of the GSU student population in every way other than by major. Approximately 
20% of the sample were found to be psychology majors and another 20% were 
English majors. The spread of SAT scores appeared to otherwise approximate the 
demographic characteristics of GSU students. Therefore, the sait5>le did provide 
a database in order to develop the model. 

The historical database contained 1,024 transcripts upon which were listed 
an unduplicated count of 2,470 discrete course numbers and grades. The sample 
of 56 transcripts with complete GRE and SAT test score information contained an 
unduplicated count of 1,065 discrete course numbers and grades. No evidence 
existed that a particular course offered in a particular year was indeed 
identical to a course bearing the same course number in another year. The 
comparability of courses in a cross^-sectional study of a single cohort of 
college seniors may be less than that of a historical group, since the potential 
differences between courses bearing the same course identification number would 
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only vary over about 4 years (from courses taken by the cohort when they were 
freshman to those completed as seniors)* A historical database accentuates the 
potential for significant changes in course structure ^ content or staffing. 
Nevertheless, for the purposes of itK>del building and preliminary analysis, 
courses of the same course nuxriber taken in different semesters or years were 
assumed to be comparable. 

Courses repeated for credit were eliminated fTcm the analysis. For 
example^ MUS 101 was found to be a performance music class. One section of thi 
class might be performance oboe, while cuiwther might be performance piano. 
Thus, students interested in music enrolled in multiple sections of the class 
during one term and enrolled repeatedly in the course Dver several terms. 
Likewise, HON 326 was found to be an honors seminar in the arts and humanities 
one quarter, in the social sciences the next quarter, and in the physical and 
life sciences yet another quarter. Therefore, these courses were eliminated 
from the analysis because they violated the assumption of conf^arability of the 
course number over the quarters represented by the historical database. 

Figure 1-1. Transcripts in the GSU Historical Database 




TOTAL DATABASE 
N Percent 



SAMPLE 



Percent 





Transcripts 



1,024 



100% 



56 



5.47% 



Unduplicated Course Numbers 



2,470 



100% 



1,065 



43.12% 



Courses Taken By 5 or More 
Students 



101 



4,09% 
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Prior research (i.e., Blackburn et al., 1976; Drees, 19B2) used a five 
percent sample of transcripts upon which to base generalizations regarding the 
undergraduate curriculum. The above analysis of the Historical Group database 
at GSU suggested that a 5 percent sanqple of transcripts would yield over 40% of 
the available courses in the college curriculum, but only about 5 percent of the 
curriculum will be represented by 5 or more students in the sanple. Therefore, 
generalizations about specific courses across a broad spectrum of the curriculum 
cannot be made based on the course- taking behavior of 5 or more students. Only 
those courses most frequently chosen by students may be included in such an 
analysis. Obviously^ average class size has a bearing on the number of courses 
available for analysis, given a specified transcript san^jle size and minimum 
number of students required in each course cell in the cluster data matrix. An 
area needing further research is that oi the relationship between sample size of 
transcripts and representativeness of the curriculum as a whole. 

The relationship between sample size 
and the college curriculum 

A persistent problem in linking the undergraduate curriculum to measures of 
student learned abilities is the number of courses from which students may 
choose. As previously mentioned, students will typically enroll in 35 to 55 
separate courses to con^lete their bachelor *s degree, although the number of 
courses vary considerably. Students select these 35 to 55 courses from a 
catalog of several thousand courses at a university or several hundred at a 
smaller college. Linking the effect of one course to the general learning of 
students therefore becomes problem^itic. 
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Two saniples of graduating seniors at Clayton State College were drawn 
during 1986-87 and 1987 -88 • However, since the individual san^^le sizes were 
snidll, a decision was made to conibine the saniples together resulting in a larger 
sample size to examine the inqportant relationships. All of the data presented 
in this report concrrns the combined sample. Comparison of the saigjle with the 
student population as a whole will be presented in Section 3. 

There were 3,427 total courses appearing on the transcripts of the 76 
students, After courses that were cross-listed or had equivalent numbers 
(through catalog changes) were identified; 1,088 unduplicated courses were 
counted on the 76 student course transcripts. Of the 1,088 courses, 177 were 
taken by 5 or more students in the Clayton State San$>le. The preliminary 
analysis, therefore, focuses on these 177 courses (most frequently chosen by 
students) . 



Figure 1-2. Suranary of Clayton State Saaople 



USABLE SAMPLE 
N Percent 



Transcripts^ GRE Tests 76 

Duplicated Clayton Courses 3,427 

Unduplicated Clayton Courses 1,088 

Nuniber of Unduplicated Courses 177 16.27% 
taken by 5 or More Students 

Nuniber of Unduplicated Courses 506 46.51% 
taken by 2 or More Students 



The question of the representativeness of a sample of courses to the total 
curriculum is exacerbated further by the lack of a precise definition of the 
total curriculum. As was evidenced by the analysis of Clayton State student 
transcripts, the exact number of courses available to students does not corres- 
pond to the college catalog. Certain courses are offered in alternate years; 
others are offered less frequently. Some courses are cancelled for lack of 
enrollment; others are split into nrultiple sections taught by different faculty 
due to large student demand. The exact number of unduplicated courses listed in 
the Clayton State curriculum database was not available at the time of this 
report. Likewise, the exact number of courses available for enrollment in any 
given year was not available. The courses in one year were not exactly 
identical to those offered the following year. What constitutes the curriculum, 
in terms of number of courses, content and variety, varies from term to term and 
year to year. 

Without an exact definition of the total curriculum available to under- 
graduates, the representativeness of a san^le of courses can only be approxi-- 
mated. Since that definition of the curriculum evolves over the peixod encom- 
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passed by a baccalaureate program, and since students enter and exit the bacca- 
laureate programs in different terms and the tenure of their undergraduate 
studies varies, the exact extent of courses from which a student can make 
choices becomes individual, nebulous and ij^precise. Even so, the data frcm the 
Clayton State Sample reflects the extent of curricular change in even the most 
frequently enrolled courses over a six-year period generally covered by the 
transcripts. More research is needed in the variability of the course offerings 
on a yearly basis. 

In the DCP Project research, sanqples of student transcripts were drawn and 
those courses enrolling 5 or more students were examined in the quantitative 
cluster analysis. When those courses enrolling 5 or more students were 
selected, the proportion of the total curriculum represented by those students 
was significantly decreased. Vihen the initial sample size is not very large, 
the representativeness of the courses to the total curriculum may be seriously 
questioned. However, many debates regarding the vitality of the undergraduate 
curriculum in producing general learning among students consider only the 
general education portion of the curriculum, not every course listed in the 
catalog. From that standpoint, the represantativeness of the courses included 
in the cluster analysis may be defined in terms of either a) the total of 
courses offered during the period of enrollment of the student, or b) the 
combinations and sequences of courses prescribed by the college or university to 
meet the general education requirements for a bachelor's degree. 

The total courses offered during the period of enrollment of a student is 
not easily ascertained at many colleges and universities. First, the 
transcripts of a cohort of students list only what the students chose, not what 
was offered but they didn't choose. Student choice of coursework is not made in 
isolation, but is made in relation to those courses not selected, those 
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previously selected, and those planned for future terms. Second, not all 
courses offered during a given period are listed in the college catalog or 
bulletin. Experimental courses and new courses, scane of which may be extended 
only in one term or year, do not appear in the catalog. Comparing the student 
transcripts with the college catalog reveals this. Thus, courses not listed in 
the catalog and not selected by a given cohort of students were among the range 
of enrollment choices available to the students. Lastly, there are courses in 
the college catalog which may not be given during the enrollment period of a 
cohort of students. While such courses were not choices to the student cohort, 
they were regarded as part of the formal curriculum of the institution. Thus, 
defining the curriculum as all courses available and/or advertised to a 
particular cohort of students may not produce an exact representation of the 
college curriculum. It may, in fact, obscure some of the most experimental and 
innovative courses which, for one reason or another, did not get recorded in the 
college catalog - 

On the other hand, if one defines the curriculum pertinent to general 
learning solely in terms of what general education courses are required for 
degree completion, the distinction between what the college intends and what the 
effects of the college curriculum are is blurred. The possibility looms large 
that a student enrollev^ in coursework that enhanced his or her general learned 
abilities but was not part of the formal general education requirements of the 
institution. Previously mentioned problems also exist with this definition of 
the curriculum as well: courses not selected are not fully represented, courses 
not listed in the catalog may be overlooked, and courses listed by not offered 
are treated as part of the range of options. In sum, the undergraduate 
curriculum is not a tidy item for analysis. 
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For the purposes of the cluster analysis, the quantitative meaning of 
course attributes is more important for generalization than the sample repre- 
sentativeness. The effect of a given pattern of coursework {its quantitative 
attributes, particularly) on a given student is more germane to determining the 
differential effect of coursework than is the representativeness of that 
coursework to the total curriculum. If one wanted a sang>le of courses 
representative of all listed in the entire curriculum, one would need a large 
enough student san^sle so that the courses e^pearing on the transcripts of 5 or 
more sample students would be representative of the total curriculum. This, 
again, presumes a means for determining the totality of a curriculum, given 
changes in courses offered over the period of student enrollment. It also 
requires an investigation of the relationship between the representativeness of 
courses to the curriculum and the relationship of the representativeness of the 
sample transcripts to the student cohort or population studied. 

However, an alternate interpretation of the relationship between course at- 
tributes and their reliability would yield a different perspective on the above 
problem. In the initial analysis of the Clayton State San^le Group, the 
criterion variables used were correlation coefficients between SAT and GRE 
scores for those enrolling in a given course. Here, the probability of error 
varies according to the number of enrolled students. When GRE item-type score 
gains are used as course attributes, however, this does not seem to be the 
case. The residual score gains calculated among all students exhibited high 
confidence levels; the regression functions of SAT scores on GliE item-type 
scores proved significant. The remaining concern, then, is the level of 
confidence attributable to a single course when it is described by the mean of 
student residual scores from the sample group who enrolled in the course. 
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At this juncture, it is important to note that the focus of the analysis is 
on courses, not students. It is not the purpose of the cluster analysis to pre- 
dict the population mean parameter of all the students enrolled in a course. 
Since the main purpose of the cluster analysis of college curriculum is to exam- 
ine the effect of an unknovm course enrollment pattern on student general 
learned abilities, the confidence level of mean residuals for an individual 
course is not of much in?>ortance because the attributes are in large part 
significantly determined by all students in the saoqple group, rather than by the 
students enrolled in that course alone. Thus, there is no reason for deleting 
those courses enrolling 4 or less students from the cluster model building 
because the course attributes are determined by student course enrollment pat- 
tern, not by the characteristic of a single course. 

The analysis of the Clayton State San^le discussed later in this report 
revealed that each coursework cluster includes a certain variety of subjects and 
levels, ranging from those intended for freshmen (100 level courses) to those 
designed for seniors (400 level courses). These clusters, derived of courses 
sorted according to the gains in general learned abilities of the students who 
took them, generate questions regarding the basic attributes of the college 
curriculum: discipline, sequence and level. 

Assumptions 

1. One learns what one studies. 

2. Courses are the primary units of le .rning in college. 

3. Transcripts are an accurate listing of the enrollment pattern of 
students. 

4. Most undergraduate courses are basically stable in content and 
instruction over time and among instructors. 
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5. The effects measured can be generalized to all the formal 
coursework in which a student enrolled. 



Limitation in analysis of curricular patterns 

The analysis of coursework patterns that lead to higher student gains in 
general learned abilities should differentiate the effect of different parts of 
the undergraduate curriculum. The analysis should also point to that curriculum 
which promises to be most effective for prcwoting cognitive develojmient. 
However, the analysis is delimited to those students who began their 
undergraduate experience at Clayton State College and attained the baccalaureate 
at Georgia State University. No analysis is presented of the differential 
effect of coursework on those Clayton State students who ended their studies 
prior to completion of their senior year at Gecriga State or who transferred to 
other baccalaureate degree-granting institutions. 

Two forms of error need to be avoided in such an investigation. One is the 
reductionist error of atten^ting to account for the variance in complex group- 
ings of coursework through individual psychological variables. In a review of 
the literature on the effects of race, gender and class on educational 
attainment. Grant and Sleeter (1986) concluded that attention to one status 
group oversimplified the analysis of student behavior, confirming the problems 
of reductionist error. The reductionist error may also occur in research equat- 
ing general learned abilities within complex academic organizations with 
intra-group cohesion and/or with individuals* identification with an academic 
discipline. The study of student learning in colleges and universities is a 
study of student behavior in such organizations, rather than the study of such 
organizations . 

A second type of error is the uniqueness-of-data approach , While it is 
important to acknowledge what is unique in each institutional learning 
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environment, this should not halt the exploration of appropriate relationships 
of curricula within different colleges and universities. This error may enumate 
from the failure to conceive of these institutions as systems (1) nested within 
and linked to larger systems (disciplinary and professional fields), and (2) 
containing smaller subsystems (departments, divisions and programs) that are, in 
turn^ linked to them (Katz, Kahn & Stacey, 1982). 

Prior research suggests that student coursework patterns found to affect 
general learned abilities can be characterized by (1) the extraneous (other than 
achievement) characteristics of the students enrolled, (2) the unique or idio- 
graphic characteristics of the learning environment, and (3) the normative 
effect of the fields of study on learning in colleges and universities (Astin, 
1970a; Pascarella, 1985). Prior research has also demonstrated that more than 
one model of college curriculum can explain the effect on student learning from 
a common set of transcript data (e.g., Hesseldenz and Smith, 1977; Kolb, 1973). 
Therefore the cluster analytic model identifies empirically'-derived course 
patterns which subsequently may be examined in terms of student characteristics 
and idiographic and nomothetic aspects of the curriculum. In this sense ^ the 
cluster-analytic model is retro-deductive in approach and is useful to the 
generation of research questions and hypotheses regarding common notions of the 
college curriculum and ius relationship to general student learning at the 
undergraduate level. 

Definition of terms and concepts 

Transfer student ; A student who has earned the equivalent of one or more terms 
of full-time work (15 quarter credits or 10 semester credits) at another 
institution of higher education prior to enrolling at the institution under 
study. 
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Hative student : A native student has obtained his/her undergraduate educational 
experience primarily from the institution under study. Native students 
entered the college or university with no more than 14 quarter credits or 9 
semester credits earned at other institutions. Native students may have 
accumulated coursework at other institutions during their bachelor's 
program {i.e., a student who attends sunaner session at another institution 
during her junior year), but such credit does not constitute a significant 
portion of their baccalaureate program. 

Graduating senior : A student who has declared his/her intention to graduate or 
is estimated to graduate during the calendar year conmencing July 1st 
andending June 30th. 

Scholastic Aptitude Test (SAT) Scores : Students may have taken the SAT examina- 
tions more than once prior to admission. When more than one set of SAT 
scores was available for a given student, the SAT score date immediately 
preceding the initiation of the baccalaureate program was used. That is, 
if a student took the exam several times and entered college in September 
1980, then the SAT scores from the test most immediately preceding 
Septeniber 1980 were used. The SAT is a precollege effects measure, and the 
more proximous the measure is to the initiation of the effects to be 
analyzed, the most desirable. 

Quarter calendar : The calendar usually consists of four ten-week terms. 

Semester calendar ; The calendar usually consists of two terms which average 
fifteen weeks each. However, each term can be as long as twenty weeks. 

Description of the GRE General Test 

The ORE General Test purports to measure verbal, quantitative and analytic 
abilities important to academic achievement (Educational Testing Service, 1988). 
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In doing so, the test reflects the opportunities and efforts of the student to 

acquire these abilities. 

Verbal abilities (GRE-V). One of the major subscores of the GRE General 

Test is that of verbal ability. Verbal ability is described as the ability to 

reason with words in solving problems. 

Reasoning effectively in a verbal medium depends upon the ability 
to discern, coit^)rGhend and analyze relationehips among words or groups 
of words and within larger tinits of discourse such as sentences and 
written passages. Such factors as knowledge of words and practice in 
reading will ... define the limits within which one can reason using 
these tools (ETS, 1988, p. 28). 

The GRE Verbal Subscore is derived from 4 types of items: analogies, 
antonyms, sentence coirqpletion and reading comprehension questions. Each is 
described below: 

Analogies (/VNA). Analogy items test students' ability "to recognize 
relationships anK:)ng words and the concepts they represent and to recognize when 
these relationships are parallel. The process of eliminating four wrong answer 
choices requires one to formulate and then analyze the relationships linking six 
pairs of words" (ETS, 1988, p. 28). 

Antonyms (ANT). Antonym items provide a direct test of the student's 
vocabulary. However, the purpose of this item-type is not merely to measure the 
student's vocabulary, but also to gage "the student's ability to reason from a 
given concept to its opposite" (ETS, 1988, p. 29). 

Reading Comprehension (RD). To successfully complete these items, students 
riUst read narrative with "understanding, insight and discrimination". These 
passages challenge a student's ability to analyze using a variety of 
perspectives "including the ability to recognize both explicitly stated elements 
in the passage and assujnptions underlying statements or arguments in the passage 
as well as the implications of those statements or arguments'* (ETS, 1988, p. 
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31). Due to the length of the narratives around which the questions for this 
item-type are built, students are given anple opportunity to assess a variety of 
relationships, such as the function of a key word in a passage, the 
relationships among several ideas, or the relationship of the author to the 
topic or the audience. 

Sentence Completion (SC). These items determine the student's ability to 
"recognize words or phrases that both logically and stylistically complete the 
meaning of a sentence" (ETS, 1988, p. 30). The student must decide which of 
five words, sets of words or phrases can best cooplete a sentence. In 
completing this type of task, the student must consider which answer gives the 
sentence a logically satisfying meaning and stylistically integrated whole to 
the discourse. 

Quantitative Ability (GRE-Q). The second subscore of the GRE General Test 
measures basic mathematical abilities, the understanding of basic mathematical 
concepts, the ability to reason quantitatively and to solve probleiM that 
require skills in mathematical analysis. The quantitative items seek not to 
exceed the abilities common to undergraduates, regardless of field of study. 
Questions test the student's facility with arithmetic, algebra and geometry. 
Questions may be in words, metric units and symbols or figures, graphs and 
tables. 

Regular Mathematics (RM). This quantitative item-type has also been 
labelled Discrete Quantitative questions and Arithmetic, Algebra and Geometry in 
various GRE and ETS publications. 

Quantitative Cons^arisons (QC). These items test the student's ability "to 
reason quickly and accurately about the relative sizes of two quantities or to 
perceive that not enough information is provided to make such a decision" (ETS, 
1988, p. 34). 
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Data Interpretation (Dl). Data interpretation items presents sets of data 
in graphs and tables and ask students to synthesize the information, choose the 
ccrect data to answer the question, or to determine that the information needed 
is not present in the data set. 

Analytic ftbility (GRE-A). The third subscore of the GRE General Test is 
designed to ineasure students' ability to think analytically. This subscore is 
con^srised of two item- types: Analytic Reasoning and Logical Reasoning. 

Analytic Reasoning (ARE). Analytic reasoning items measure a student's 
ability "to understand a given structure of arbitrary relationships among 
fictitious persons, places, things, or events, and to deduce new information 
from the relationships" (ETS, 1988, p. 38). 

Logical Reasoning (LR). These items assess a student's ability to 
understand, analyze and evaluate positions and contentions. Specific questions 
may evaluate a student's ability to recognize a point of argument or the 
assumptions on which a position is based, to draw conclusions or form 
hypotheses, to assess the manner of argxanents and the evidence supporting them. 

While the GRE General Tests are designed to describe the student's broad 
verbal, mathematics and ajialytic abilities, the 9 individual item-types of the 
Test provide discrete measures of general learned abilities. One should avoid, 
however, making the assumption that the GRE measures all gener.^^l learned 
abilities associated with collegiate learning or even those intended as the 
educational goals of a particular college, university, program, major or course. 
Nevertheless, the GRE provides a broad set of measures of general learning from 
which a model to assess selected gains in cognitive development of college 
students. 
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Suntmary 

In this section the purpose, scope^ and method of the DCP Project were 
described. The purpose of the research was to test the hypothesis that student 
enrollment in different patterns of coursework affects the development of their 
general learned abilities. 

There was no clear consensus in the literature reviewed as to what 
constitutes the general learned abilities of college students. There was 
general agreement that the assessment of students* general cognitive development 
necessitates multiple measures of general learning. 

One set of such measures consists of the 9 item-types of Ihe general test 
of the Graduate Record Examination. The GRE General Test has been criticized as 
an assessment measure of baccalaureate learning because of the limited number 
and advanced abilities of the students upon which the tests were normed* 
Similarly, the multiple choice format of the GRE has been criticized for not 
measuring higher-order reasoning and creative thinking skills. These criticisms 
notwithstanding, the GRE presents a common set of measures of general collegiate 
learning. 

The strong correlation of the GRE with the Scholastic if^ptitude Test (SAT) 
affords an opportunity to control the assessment of student learning for the 
comparable knowledge, skills and abilities students possessed upon admission to 
college. The largest amount of variance in general learned abilities logically 
should be attributable to learning prior to college. Nevertheless, when such 
learning is removed from the analyses, residual scores should provide indicators 
of students' development during the college years. 

Just as determining what constitutes student achievement in college is 
problematic, so too is the determination of what the formal curriculum of a 
college or university is. A distinction in the DCP Project is made between the 
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intended curriculum^ as stated in the college catalogs and bulletins and the 
actual curriculeur record, as represented on the student transcript. A 
coursework pattern was defined as a snt of courses whose effects on general 
learning are sijnilar. This definition is an unpirical artifact of student 
enrollment behavior, rather than an academic plan or stated curricular sequence, 
as might be found in a catalog or program brochure. 

By clustering courses into patterns according to their effect on general 
learned abilities, a basis is provided for examining what students took in light 
of what they learned. what the GRE measures was what the college intended as 
the outcomes of the general education curriculum, and if the course shown to 
affect positively the general learning of undergraduates were the same as the 
colleges general education requirements, then that college may take pride in the 
evidence of the effectiveness of its curriculum. But such a comparison of what 
a student takes with what that individual learns is predicated upon the 
hypothesis that enrollment in a different pattern of coursework leads to 
different effects in general learning. 

To test the differential coursework hypothesis, a cluster analytic model 
was developed. The residual scores of GRE item- types were attributed to the 
specific coursework in which students enrolled. Each course was described in 
terms of the mean of residuals of the students who had enrolled in the course. 
Thus, there were 9 mean item-type residuals for each course found on student 
transcripts. Next, courses were sorted and clustered according to these 9 
criterion variables. Finally, the validity of the groupings was tested using 
discriminant analyses. From the discriminant analyses, coursework affecting 
general learning could be differentiated from that serving some other role (such 
as learning within the major or learning not measured by the GRE). 
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Through the development of the cluster analytic model, the effect of 
coursework on general learned abilities may be assessed. Furthermore, the 
extent to which the item-types of the GHE represent discrete measures of general 
learning can be assessed. Similarly, the intentions of a college or university 
general education curriculum can be cca^jared to en^sirically-derived coursework 
clusters found to be associated with gains in general learning. Thus, the model 
can be used to assess student learning to determine the strength and 
independence of the measures of learning selected, and to ccaqsare the intended 
curriculum with the actual course-taking behavior of students. 
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n. Methodology and Procedures: 
Cluster Analytic Model 

A conceptual/en5>irical approach was used in the selection, testing and 
adoption of a specific methodology for the analysis of coursework patterns. The 
approach was conceptual in that theoretical concepts differentiating coursework 
discussed in the previous section restricted the aapirical approach to conform 
to the nature and orientation of the research problem. What follows is a dis- 
cussion of the process of cluster analysis and its application to the investiga- 
tion of coursework patterns; in that discussion, cluster analysis is contrasted 
to other statistical methods of potential value to the research investigation. 

Previous transcript analysis studies have used the general linear model and 
regression analysis (Benbow and Stanley, 1980, 1982; Pallas and Alexander, 1983; 
Prather and Smith, 1976a, 1976b). The rationale for the use of regression is 
based upon practical and theoretical justifications. Regression analysis allows 
maximum design flexibility and is statistically robust. Transcript analyses 
involve large amounts of data. For example, Prather et al. (1976) examined 
8,735 student transcripts which collectively contained 189,013 individual course 
grades. Regression analysis provides an effective technique for presenting the 
diverse nature of the data while maintaining a consistent analysis rationale. 
However, the general linear model does not provide a direct means of assessing 
the additive and ten5x>ral aspects of course patterns, as described in the pre- 
vious chapter. 

To distinguish cluster analysis from other approaches, certain terms need 
definition. The term classification is used here to refer to the categorization 
of the courses in which students enrolled over the duration of their baccalaure- 
ate program. It is the systematic and unique way a college or university labels 
and arranges its courses (i.e.. Honors 101, French 340, etc.); that scheme or 
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arrdngement of classes is already known in a disaggregate form on student tran- 
scripts. Identification is the allocation of individual courses to be 
RStablished in categories on the basis of specific criteria (i.e., Biology 205 
is classified by many universities as a sophomore level class in the department 
of Biology). 

Discriminant analysis is a process used to differentiate between groups 
formed on an a priori basis (See Biglan, 19738 for an example). Discriminant 
analysis does not discover groups; it identifies a set of characteristics that 
can significantly differentiate between the groups. The process allows the 
analyst to allocate new cases to one of the a priori groups with the least 
amount of error. In contrast, cluster analysis recovers groups representing 
particular patterns from diverse populations (U5rr, 1983; Romesburg, 1984). In 
the model developed to analyze coursework patterns, cluster analysis is used to 
classify courses according to student achievement criteria, while discriminant 
analysis is used to test and provide secondary validation of the cluster 
groupings and to identify those criteria which significantly differentiate one 
cluster of coursework from another. 

Factor analysis is different from cluster analysis in that its attention is 
on the similarity of the variables (attributes). The aim is to identify a small 
number of dimensions (factors) that can account for individual differences on 
the various measures or attributes. Thus, the aim of factor analysis is to 
reduce or consolidate the number of attributes of a variable set while the pur- 
pose of a cluster analysis is sin^Jly to classify or taxonomize data into groups 
on the basis of a set of attributes. Miller (1969) examined 48 common nouns; 
through cluster analysis he identified five subgroups referring to living 
things, nonliving things, quantitative terms, social interactions, and 
emotions. Another exairple of cluster analysis is Paykel's (1971) analysis of 
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165 depressed patients. Using synqptom ratings and historical variables^ he 
grouped the patients into four clusters: the retard psychotic, the anxious, the 
hostile, and the young depressive. Cluster analysis refers to a wide variety of 
techniques used to classify entities into homogenous subgroups on the basis of 
their similaurities. 

The end products of cluster analysis are clusters or pattern sets. Since 
the exact number and nature of the course patterns is not known in advance, the 
clustering process is actually technically preclassif icatory. In other words, 
cluster analysis techniques are used to construct a classification scheme for 
unclassified data sets. In this way, cluster analysis enpirically arranges the 
courses of a college curriculum using student decision-making behavior (as rep- 
resented on transcripts) as the primary source of information. The courses are 
classified in a hierarchical dendrogram or tree. The relationship between 
courses is determined by their similarity on the criteria used in the classifi- 
cation. In this way, the similarity between courses is determined empirically, 
rather than by arbitrary concepts (i.e., "life sciences") or levels (i.e., 
"freshmen level survey"). This conceptual/ en^sirical approach was selected due 
to the lack of agreement in the higher education literature on a common research 
paradigm, model or philosophy for the organization of coursework (Bergquist et 
al., 1981; Biglan, 1973a; Furhmann and Grasha, 1983; Gaff, 1983; Rudolph, 1977; 
Sloan, 1971; Veysey, 1973). 

Cluster analysis conforms to the conceptual restrictions placed on the mod- 
el in order to assess the effect of coursework patterns on student learning. 
Cluster analysis provides a statistical procedure for examining coursework using 
multiple criteria. It can accommodate both quantitative and qualitative attri- 
butes of varying dimensions. Thus, the criterion selected need not be test 
scores; nominal, order, interval and ratio data have been successfully used as 
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attributes in cluster analysis (Romesburg, 1984). Cluster analysis uses these 
attributes to arrive at patterns of coursework independent of any institution- 
ally prescribed a priori distinctions among courses. It therefore is capable of 
testing notions of coxnbinations, sequence emd progression of courses within the 
college curriculum. It leads to the discovery of clusters (or patterns) of 
coursework in student transcripts^ based on the attributes of students"* general 
learned abilities. Since the purpose of the cluster analytic model is to group 
coursework hoinogeneously according to its relation to student learning outcomes 
(Lorr, 1983; Romesburg, 1984)^ cluster analysis was chosen as the primary 
methodology for analyzing student transcripts in this Differential Coursework 
Patterns Project. 

General procedural steps 

This section describes the steps in the statistical process embodied in the 
cluster analytic model. These steps address the four research objectives 
(discussed in Section 1) of the model. Student score gains are derived. 
Student transcripts are examined^ and courses reported on them are clustered 
into patterns based on the score gains of the students who enrolled in the 
courses. Resulting patterns of coursework are again analyzed and classified 
according to attributes associated with student course-^selection and 
decision-making- Resulting patterns of coursework are analyzed and classified 
according to attributes associated with the educational environment of the 
college or university: (a) the type of college or university, as indicated by 
Carnegie classification (1981), (b) the type of general education degree re- 
quirements of the institution, as indicated in the college catalog or bulletin^ 
(c) the type of academic discipline or field of study, as indicated by the 
course prefix on the transcript, and (d) the student demographic character- 
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istics, as indicated on the denK>grap>ic questionnaire con5>leted by the student 
at the time of GRE testing. Hypothesized patterns of coursework generated from 
one set of student transcripts may be validated through the replication of the 
cluster analytic model to a second san^le of student transcripts. 

Quantitative versus qualitative measures 

As previously described in Section 1, there is more than one view of what 
constitutes representation of a college or university curriculum within a sample 
of student transcripts. One view suggests that only those courses in which 
students most frequently enroll constitutes the curriculum associated with gen- 
eral learning in the undergraduate program. A second view holds that any course 
offered may contribute to the general learning of students. The first view im- 
plies a more restricted view of the curriculum than does the second. These con- 
trasting views resulted in the development of two alternate procedures for as- 
sessing the associated effects of coursework patterns on general learned abili- 
ties. Reported in the following section are the results of the first procedure, 
the quantitative cluster analysis . The second procedure, qualitative cluster 
analysis , is described thereafter. 

The cluster analytic model uses multiple measures of general learned abili- 
ties as attributes with which to classify courses taken into patterns. These 
attributes can be expressed quantitatively or qualitatively. For exanple, a 
sophomore level mathematics class (i.e.. Math 201) can be described according to 
the mean score gain of students (from the sample) who enrolled in the course. 
Math 201 can also be described nominally; here the researcher simply notes 
whether one or more students with high score gains enrolled in the course. Both 
the quantitative and qualitative descriptions of Math 201 serve to determine the 
relation of the course to other courses according to the item-type criteria 
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variables. 

a sanple of students is used to examine the effects of a particular 
college curriculum on general learning, there are a limited n\imber of courses 
within the curriculum which can be analyzed quantitatively. The GSU Historical 
Group exaii5>le, previously described, illustrated this problem. Only a limited 
percent of all courses appearing on the sample transcripts can be analyzed if 
the number of students enrolling in a given course is a concern in th© analysis. 
However, such quantitative analysis of the curriculum can yield much more accu- 
rate information regarding the effect a particular course may have on a given 
measure of student general learned ability. To generalize about a course on the 
basis of 5 or more student score gains provides a level of information that far 
exceeds that of sin^sly noting whether any student who performed well on a given 
measure enrolled in that course. 

There are advantages emd disadvantages to either the quantitative or the 
qualitative approach. In the quantitative analysis, a limited number of courses 
can be examined, but, in practice, those courses are those in which iWDSt stu- 
dents enroll and encortgsass all those in which students are required to enroll. 
Math 101, a required mathematics course in a college's curriculum, would be 
included in those courses examined in a quantitative cluster analysis since all 
students are required to enroll, while Math 450 designed primarily for senior 
level math majors would not be included — assuming the san^^le of students is 
random and not confined to mathematics students. 

There are those, however, who may argue that it is the advanced coursework 
within a given discipline which facilitates general student learning. It has 
been suggested that the study of liberal arts disciplines teaches students a 
mode of inquiry which facilitates their learning of other forms of knowledge, 
abilities and skills (Biglan, 1973a, 1973b). Similarly, courses with tradition- 
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ally restricted enrollments may not appear in an analysis of coursework selected 
by the frequency of enrollment. Analysis of the effect of credit for study 
abroad or honors programs or the assessment of coursework patterns of specific 
groups of students might not be possible. Therefore, under these and related 
circumstances, it is desirable also to examine as many courses of a tudent's 
transcript as possible, rather than restricting the analysis to only those 
courses in which students most frequently enroll. 

Examination of all courses on a student's transcript may not be feasible. 
Some courses may have only one student enrolled from the saiii>le group, the 
cohort, or population of students examined. Recall that in the cluster analytic 
model, a student's GRE item- type residuals are attributed to all the courses in 
which he/she enrolled. The contribution of individual courses to the curriculum 
is calculated as the sum of the effects of the students who enrolled in those 
courses. Courses with low enrollments frcan the sample group or the group being 
examined have higher margins of error because the effects are discerned from a 
smaller number of students. Thus, courses with an enrollment of one student 
from the sample group do not provide a basis for quantitative analysis, while 
courses with limited enrollment (2 or more) may be amenable to the treatment of 
that enrollment solely as a nominal variable. 

In a quantitative cluster analysis, the metrics used for each course are 
the mean GRE item- type residuals which contain interval information about the 
gains of students who enrolled in the course. In a qualitative cluster 
analysis, the metrics used are whether students with high score residuals did or 
did not enroll; the metric is reduced to a dichotonous ncxninal variable. There 
is a trade-off in a qualitative cluster analysis between inclusiveness of the 
curriculum and precision of the information. 
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Any quantitative attribute, such as a GRE item-type residual, can be di- 
chotomized and converted into a binary attribute (Anderberg, 1973). Such a pro- 
cedure lessens the precision of information in the data set because the process 
is irreversible. The data from an interval scale is collapsed into a nCTiinal 
one. It is conanonly held that ratio scales provide more precise information 
than interval scales, that interval scales are more precise than ordinal ones, 
and that all the preceding are more informative than nominal scales. However, 
the choice of scales is constrained by different factors. 

First, institutional researchers are often under monetary constraints. The 
costs of obtaining test scores for all college graduates, for exan^le, may not 
be feasible on an on-going basis. Hence, it may not be practical to gather the 
number of student transcripts and assessment information needed to use the quan- 
titative cluster analysis with courses other than those in which students most 
frequently enroll. 

Second, institutional researchers have a choice between an intensively de- 
tailed picture of the curriculum using the ratio data of mean residuals or a 
less detailed picture provic'ed by binary information. As has been previously 
discussed, there are occasions when the scope of the analysis is to be preferred 
over the precision of the analysis. 

Third, "data do not automatically inform the researcher" (Romesburg, 
1984). To have meaning, transcript and test data must be interpretable within a 
curricular context. The primary question is, "Which coursework patterns 
contribute to general student learning?" The secondary questions are, "How much 
do the patterns contribute?" and "What is their relative contribution?" 
Qualitative analyses are not categorically inferior. In this case, a 
qualitative analytic question precedes the one which may be answered 
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quantitatively . 

Procedure 1: Quantitative cluster analysis 

Described below are steps required in Procedure 1 (Quantitative Analysis) 
to assess the effects associated with the coursework patterns on the general 
learned abilities of college students. The research design uses as data sources 
transcripts and GKE and SAT test scores from a sanple of students. The 9 item- 
type categories of the General Tests of the Graduate Record Examination are used 
as measures of general learned abilities of college seniors. These seniors' SAT 
scores are used as variables to control for the academic abilities of these 
students when they first entered college. The student tramscripts are used as 
the record of the sequence of courses in which these seniors enrolled. 

The first objective of the cluster analytic model is to determine the stu- 
dent gains in general learned abilities over the time of their baccalaureate 
program. To do this, first the residual score of each item-type for each 
student is calculated; the residual score is the difference between the 
student's actual score and the score predicted by the student's corresponding 
SAT score. Thus, for each student outcome measure there is a student score gain 
for each person in the san^le group. 

The second objective is to determine patterns of coursework on the student 
transcripts which are associated with student score gains. This is accongjlished 
through cluster analysis, using student score gains (GRE item-type residuals) as 
attributes of the courses in which students enrolled. 

A raw data matrix consisting of columns of courses and rows of score gains 
is created. The aean residual score for all the gtudopts in the saaple who 
enrolled in a given course is calculated and becoaes the Metri c value for that 
course. The correlation coefficient is used as the reseablance coefficient to 
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transform the data matrix into a resetriblance matrix, wherein the similarity of 
residual scores for students enrolling in one course can be coB?>ared with those 
enrolled in another course. Once the resegblance natriac indicating the propor- 
tional relationship of courses is established, a clustering method is selected 
and executed to arrange a tree or dendrogram of courses related by the student 
score gains. Next, a discriminant analysis is p^^rfonned on the resulting 
clusters of coursework to (a) determine the extent to which the courses have 
been correctly classified according to the 9 mean student residual scores, (b) 
to determine which of the 9 mean residual scores were correlated with particular 
discriminant functions, and (c) to determine which coursework clusters exhibited 
high mean residual scores relative to each discriminant function. From the 
discriminant analysis an association can be inferred between coursework patterns 
(clusters) and general learned abilities (student score gains on 9 criterion 
variables). The cluster-analytic procedure groups courses frequently chosen by 
students according to the strength of their associated effect on the student 
score gains. 

Described in greater detail below the steps followed in this cluster ana- 
lytic procedure: 

Step 1 . Calculate a student residual score for each item-type (attribute) of 
each student GRE. This step removes the predictive effect of the 
student's SAT scores from the GRE item-type, thereby controlling for the 
academic ability of the student upon entrance to college. For GRE 
Quantitative item- types, the effect of the student's SAT Math score is 
partialled out. For the GRE Verbal item-types, the effects of the SAT 
Verbal score is partialled out. For the GRE Analytic item-types, the 
effect of the combined SAT Verbal and SAT Math scores are partialled 
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out. In this way, the student's academic abilities prior to entering 
colloge is controlled when calculating stucunt residual scores. 

Step 2 . Calculate the mean residual score for each course enrolling 5 or more 
students from the sainple group. Cross-listed courses are standardized 
so that they have only one identifier. Cross-listed courses include 
those with identical munbers that have different labels. Courses with 
the same course identifier but with abstentiously different content 
(i.e., "Music 101: Voice" and "Music 101: Piano") are excluded from the 
analysis. However, catalog changes are accounted for. If Math 201 in 
1982 was renumbered as Math 211 in 1985, Math 201 and Math 211 for those 
years are treated as the same course for the purposes of analysis. 

The proportion of courses included in the analysis is related to (a) the 
extensiveness of the course listings in the curriculum, and (b) the size 
of the student san^le. The more extensive the curriculum, the less 
frequently 5 or more students from the sample will have enrolled in the 
same course. Likewise, the smaller the size of the student sample, the 
less frequently 5 or more students from the sample will have enrolled in 
the same course. 

Step 3 . Create a raw data matrix by using the mean residual scores for the 

courses found on 5 or more of the student transcripts. The rows in the 
data matrix consist of the 9 GRE item-type scores while the columns 
represent those courses enrolling 5 or more students. Each cell value 
of the matrix is a mean GRE item-type residual score for those sanplc 
group students enrolling in a specific course. For exan^le, the course 
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{object) in the first column in the data matrix is ANTHROPOLOGY 101, and 
the student outcome measure in the first row of the data matrix is DATA 
INTERPRETATION. The student score gains are .40, .45, .50, .55, and 
.60; the mean score gain, therefore, is .50 and is entered as the metric 
variable in cell (1,1) of the matrix. Since the variables in each row 
are of the same magnitude, and therefore, have con^jarable effect on the 
resulting cluster analysis, the data matrix does not need to be 
standardized (Romesburg, 1984). The cluster analysis will taxonomize 
courses in the curriculum according to whether students who showed 
positive residuals on each item-type were enrolled in the courses. This 
step prepares a raw data matrix to be used in a general cluster analysis 
based on quantitative data. 



Step 4 . Select a resemblance coefficient. The resemblance coefficient 

(Komesburg, 1984) is also called the similarity index (Lorr, 1983). The 
purpose of the resemblance coefficient is to explain the similarity (or 
dissimilarity) of each cell to each of the other cells in the data 
matrix; it is expressed mathematically. There are many resemblance 
coefficients; each will express the similarity between courses (objects) 
in a slightly differently way. Each coefficient is appropriate for 
achieving slightly different research goals. 

The resemblance coefficient selected for this study is Pearson's 
product -moment correlation coefficient. It is appropriate for use with 
ratio data. The resemblance coefficient indicates the similarity of 
courses to each other according to the 9 item-type residuals 
(attributes) coded in the data matrix. The resemblance coefficient 
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expresses the relationship of tvfo courses proportionally. 



Step 5 . Calculate a resemblance matrix from the raw data matrix. The 

resemblance matrix is calculated by transforming the raw data matrix 
using the correlation resemblance coefficient. In the cluster analytic 
model, the data matrix consists of quantitative data described by 9 
attributes ranging in value from 1.00 to -1.00. In the resemblance 
matrix, the columns represent the first course (object) in a pair, the 
rows represent the second course (object) in a pair. The resemblance 
coefficient (Pearson's r) is entered into each cell. The cell value 
represents the extent to which the attributes on the first course 
explain the variance in attributes on the second course. The 
resemblance coefficient serves as a measure of similarity between one 
course and each other course in the calculation of clusters or 
coursework patterns. 

Step 6 . Select and execute the clustering method. A resemblance matrix is 
transformed into a tree of related courses (objects) b\ use of a 
clustering method which is a series of steps that removes values from 
the resemblance matrix. Therefore the size of the matrix is reduced. 
Each time a value is removed from the resemblance matrix it is placed in 
the cluster tree or dendrogram. In the last step, the resemblance 
matrix disappears corrpletely and the tree is coiiqjleted as the last value 
is inserted. 

Rdesburg (1984, p. 139) recommends the unweighted pair-group method 
using arithmetic averages (UPGMA), also known as the average linkage 
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method. UPGMA is reconmended over single linkage clustering method 
(SLINK) and COTiplete linkage clustering method (CLINK) for two reasons. 
First, it can be used with any resCTiblance coefficient, while SLINK and 
CLINK are designed to be used with interval and ratio data in a 
quantitative data matrix. Second, it judges the similarity between 
pairs of clusters in a less extreme manner than do SLINK and CLINK. The 
average linkage method (UPGMA) is available on SPSSx, SAS and BHDP 
statistical packages. 

Step 1 . Determine the optimum number of coursework clusters. Cluster analysis 
is a procedure for taxonomizing or classifying coursework data. The 
number of groups or patterns in which the data is classified according 
to the criterion variables is an arbitrary one. Once relationships 
between courses have been determined, the researcher must decide on how 
many groups in which to put the data. Discriminant analysis provides a 
means to test the secondary validity of the coursework pattern 
groupings. 

By conputing successive cluster analyses for different numbers of 
clusters and then conducting discriminant analyses on the resultant 
groupings, one can identify the number of clusters which has the highest 
predictive value, given the criterion variables used. Using the DIS- 
CRIMINANT program in SPSSx, for exan^le, will identify how many members 
of each coursework pattern or cluster were correctly classified, how 
many could be classified in other patterns, and what was the overall 
percentage of correct classification. 
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The number of clusters with the highest predictive value may not be the 
sole objective in examining the merits of different cluster solutions to 
the cluster analysis. Theoretically, a four cluster solution may have 
high predictive value for GRE it«n-types because the item-type residuals 
are forced into three discrijidnant functions which should approximate 
the GRE sub-scores. Likewise, a 10 cluster solution may prove to be 
slightly less predictive, but the 9 GRE item-type residuals may be iiKsre 
clearly associated with discrete coursework patterns. Careful visual 
inspection of the cluster dendrogram often suggests appropriate cluster 
solutions to test using discriminant analysis. 

Step 8 . Determine which criterion variables contribute significantly to which 

discriminant functions. DISCRIMINANT in SPSSx, for example, calculates 
the pooled within-groups correlations between the discriminating 
variables (in this case, the mean residual scores on the 9 item- types) 
and the canonical discriminate functions. Large positive and negative 
correlations are identified. Eigenvalues for each discriminant function 
are assessed. Eigenvalues express the proportion of variance in student 
scores explained by the discriminant function. Discriminant functions 
that explain less than 5 percent of residual score variance or that have 
a probability of error exceeding .001 are discarded. Next, the group 
means for each coursework cluster can be examined. In this manner, the 
patterns of coursework associated with one or more mean item-type 
residual scores can be identified. 
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step 3 . Repeat Steps 1 to 8 using a second cohort of students • Following Steps 
1 through 8 will produce a set of hypothesized relationships between 
coursework patterns and student score gains on 9 criterion measures of 
general learned abilities. Hypothesized relationships cannot be tested 
or validated using the same data* Therefore, a second institutional 
sample is drawn. A second group of students are tested and a second set 
of transcripts and student score gains are evaluated. Repeated use of 
the model should refine and clarify members within each coursework 
pattern. 

Through the above 9 steps, the cluster analytic model classifies the most 
frequently enrolled courses according to their associated effect on student 
score gains. Procedure 1 classifies courses according to a ratio index of 
similarity to other courses. While Procedure 1 may examine only a fraction of 
all the courses in a college curriculum, it does provide a means to 
differentiate the effect of required courses or courses in which roost students 
enroll- For exan^le, in the GSU historical database used in model-building and 
testing, a five percent sample of student transcripts enabled an examination of 
only five percent of courses appearing on those transcripts (the percentage of 
courses enrolling 5 or more students from the san^le group). However, the 
courses examined in that 5 percent corresponded closely to those courses 
identified as meeting the College's distributional degree requirements in 
general education. 

Procedure 2: Qualitative cluster analysis 

While the Quantitative Cluster Analysis Procedure (Procedure 1) examined 
those courses enrolling 5 or more students from Clayton State Samples, it may be 
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desirable to associate a greater proportion of the curriculum with the students 
who showed significant intprovement on one or nwre of the GRE item-types. A 
second Cluster Analytic Procedure was developed. Procedure 2 was labeled 
"Qualitative Cluster Analysis". In Procedure 2 only, courses enrolling 2 or 
more students with positive residuals were considered. Due to the small number 
of students in the Clayton State Sanqples, Procedure 2 was not utilized. It is 
described here for information purposes only. 

The qualitative cluster analytic procedure described in preceding progress 
reports (Ratcliff, 1988a, i988b) dichotomized the student sanple into subgroups 
of students: those scoring at or above the mean on a given attribute and those 
scoring below the mean. Separate cluster analyses were then conducted on each 
sub-group and the results of the two were then coirpared. It was discovered that 
such a procedure tended to exclude the courses with the highest enrollment 
levels, since the higher the enrollment, the greater the probability that 
students from both the high-residuals group and the low-scoring residuals group 
enrolled in the course. Conversely, in low enrollment classes, the probability 
that students from both groups enrolled was markedly reduced. For this reason, 
further testing and development of the qualitative cluster analytic procedure 
was warranted. 

The steps required in Procedure 2 to classify coursework patterns based on 
student residual scores in general learned abilities are described below. As in 
Procedure 1, student GRE item- type residuals first are con?>uted for each student 
outcome measure. Second, those students who scored at or above the mean of 
student residuals for a given item-type are identified. Third, the proportion 
of students with residuals at or above the mean relative to the total student 
sample is computed for each item-type for each course. The standard error of 
estimate for that proportion is calculated. The metric for each attribute 
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(item-type) for each course thus becomes the proportion of students enrolling in 
the course whose residuals are at or above the mean weighted by the reciprocal 
of the standard error of estimate for that proportion. In cases where the 
standai-d error is zero, a constant--en5>irically determined- is used to avoid the 
mathematical problem of dividing by zero. 

Once the metric of the weighted proportion of high-residual students 
enrolled in a given course is calculated for each item-type, a raw data matrix 
consisting of columns of courses and rows of weighted proportions is created. 
The weighted proportion of hiah-residaal students in the sample becomes the 
metric value for that course. The correlation coefficient is used as the 
resanblance coefficient to transform the data matrix into a esemblance matrix. 
In the resemblance matrix the similarity of proportions of high-residual 
students enrolling in one course can be coirf>ared with those enrolled in another 
course. Once the resenblance matrix of courses is established, then a 
clustering method is selected and executed to arrange a tree or dendrogram of 
courses related by the student residual scores. Next, a discriminant analysis 
is performed on the resulting clusters of coursework to (a) determine the extent 
to which the courses have been correctly classified according to the 9 mean 
student residual scores, (b) to determine which of the 9 mean residual scores 
were correlated with particular discriminant functions, and (c) to determine 
which coursework clusters exhibited high mean residual scores relative to each 
discriminant function. From the discriminant analysis, an association between 
coursework patterns (clusters) and general learned abilities (student residual 
scores on 9 criterion variables) can be inferred. The qualitative 
cluster-analytic procedure groups courses according to the weighted proportion 
of students with high residuals. Thus, courses with larger proportions of 
students demonstrating at or above average gains are classified together. 
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Below are the steps in Procedure 2 of the cluster analytic model: 

Step 1 . Calculate a residual score for each item-type (attribute) of each 
student GRE. This step is identical to that of Procedure 1 and 
removes the predictive effect of the student's SAT scores froea the 
GRE item-type, thereby controlling for the academic ability of the 
student upon entrance to college. 

Step 2 . Con5>ile the residual score of each student at or above the mean for 
each item- type (attribute). Calculate the proportion of students 
at or above the mean enrolling in each course and the standard 
error of estimate for that proportion. Weight the proportion of 
students at or above the mean for a given course and a given 
item- type by the reciprocal of the standard error of estimate for 
that proportion. Exclude courses with an enrollment of 1 or no 
students, since a standard deviation cannot be calculated from an 
enrollment of one. 

Step 3 . Construct a data matrix with columns consisting of all the courses 
(objects) with an enrollment of 2 or more students, as evidenced by 
the transcripts of the students in the sample. The rows denoted 
each of the 9 item- type residuals. The cells in the data matrix 
consist of the weighted proportion of students whose residual 
scores were at or above the mean for the given item-type 
(attribute). Cross-listed courses are standardized so that they 
have only one identifier. 
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s tep 4 > Select a resemblance coefficient. The same reseiriblance coefficient 
used in Procedure 1 can be used in Procedure 2: Pearson *s 
product-iroment correlation coefficient • This will indicate the 
similarity of courses to each other according to the weighted 
proportion of students at or above the mean on each of the 9 
item-type residuals (attributes), as coded in the data matrix. 



Step 5 > Calculate a resemblance matrix from the raw data matrix. As in 

Procedure 1, the resemblance matrix is calculated by transforming the 
raw data matrix using the correlation resemblance coefficient. The cell 
value represents the extent to which the weighted proportion of students 
at or above the mean on the attributes in the first course explain the 
variance in the weighted proportion of students at or above the mean on 
the attributes of the second course. Thus, the resemblance coefficient 
serves a measure of similarity between one course and each other course 
in the calculation of clusters or couxsework patterns. 



Step 6 t Select and execute the clustering method. A resemblance matrix is 
transfo:rmed into a dendrogram or tree of related courses (objects) 
by the use of a clustering method which is a series of steps that 
removes values from the resemblance matrix. Therefore, the size of 
the matrix is reduced. Each time a value is removed from the 
resemblance matrix it is placed in the tree. In the last step, the 
resemblance matrix disappears ccxupletely and the tree is completed 
as the last value is inserted. 
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As in Procedure 1, the unweighted pair-group method using 
arithmetic averages (UPGMA), also knovm as the average linkage 
method, is used as the clustering method. Recall, UPGMA is 
available on SAS, SPSSx, and BWDP statistical packages. 

Step 7 . Determine the optimum number of coursework clusters. As in Procedure 1, 
discriminant analysis is used to provide a means to test the secondary 
validity of the coursework pattern groupings. 

Also as in Procedure 1, the number of clusters with the highest predic- 
tive value may not be the sole objective in examining the merits of dif- 
ferent cluster solutions to the cluster analysis. Cluster groupings 
should be sought which most clearly disclose the relationship between 
the criterion variables and the coursework patterns contained in the 
cluster tree. Careful visual inspection of the cluster dendrogram often 
suggests appropriate cluster solutions to test using discriminant 
analysis. 

Step 8 . Determine which criterion variables contribute significantly to which 
discriminant functions. This step is identical to that in Procedure 1. 

Step 9 . As in Procedure 1, repeat Steps 1 to 8 using a second cohort of students 
to determine if the patterns found in the first cohort can be replicated 
in the second. 

Through the above 9 steps of Procedure 2, the cluster analytic model clas- 
sifies all courses with an enrollment of 2 or more students (from a sangple of 
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transcripts) according to the proportion of students enrolling who evidenced 
gains at or above the mean on the selected measures of general learned ability. 

Procedure 2 allows for the examination of a greater proportion of the cur- 
riculum than Procedure 1, but in doing so, it reduces the precision of informa- 
tion used in the analysis through the transformation of quantitative data on 
student residuals into a qualitative dichotomy (between those students whose 
residuals were at or above the mean and those whose residuals were not). In 
Procedure I, over one-third of the curriculum appearing on the transcripts of 
the Clayton State San^jles #1 and #2 were analyzed. Due to the small number of 
students in the Clayton Samples, Procedure 2 was not used. 

Describing the resulting coursework patterns 

The third objective of the cluster analytic model is to describe the pat- 
terns of coursework resulting from Procedure 1 analysis according to sequences 
and combinations of courses within the cluster, according to term of enrollment 
data found on student transcripts. Can the coursework patterns resulting from 
the Procedure 1 analysis be meaningfully described by the above factors? Every 
course in each resulting coursework pattern can be described according to any of 
the above factors, just as it was described by associated student residual 
scores. The extent to which courses in one cluster differ from other clusters 
on any of the factors can then be tested using analysis of variance (ANOVA) or 
the Kruskal-Wallis one-way analysis of variance for ordinal data. 
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III. Description of the Clayton State College Combined Samples 
This section describes the Clayton State combined samples (1986-87,1987-88) 
group of graduating seniors. The extent to which this ean^sle is analogous and 
representative of the population of Clayton State seniors is assessed. 

Eligibility Requirements 

Three criteria were established to determine eligible students from Clayton 
State to participate in this study. Students had to meet all three eligibility 
requirements in order to participate in the study: 

1. a graduation date of May 1987 or an expected graduation 
date of December 1986 or Hay 1987 (these criteria are for 
inclusion in sanqsle #1, the same criteria, except one year 
later, are used for sample ^2); 

2. at least 84 total accumulated credits; and 

3. no less than 5 transfer credits from Clayton State. 

Any students with fewer than 84 total accumulated credits were e'xcluded 
since, despite their "expected graduation dates" listed in the College's 
database, they would not be able to graduate by May 1987 (or May 1988 for second 
sample) taking normal course loads. Students with less than 5 transfer credits 
were excluded since such students had not experienced a reasonable portion of 
their undergraduate education at Clayton State. 

Combined Samples #1 and #2 
Clayton State combined sangsles consisted of the transcripts and test scores 
for 54 students from san^sle #1 and 22 students from saiaple #2. While the 
Clayton State san^jles did approximate the characteristics of graduating seniors 
for that year, several minor variations between the combined samples' student 
characteristics and those of the population as a whole are reported below. 
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Gender has been shown to be a significant factor in the academic 
performance of college undergraduates. Nearly two-thirds (65.8%) of the sarnple 
were female, while 55.4 percent of the population were female (see Figure 3-1). 

Race and ethnicity also have been shown to be strong predictors of academic 
performance. Ninety- two point eleven percent of the combined San^ile were white, 
while 94.9 percent of the population of graduating seniors were also white (see 
Figure 3-2). 

Major field of study has been shown to be correlated to performance in the 
GRE exa.ninations. The distribution of majors in the combined Sample 
approximated that of the population. Majors in Early Childhood Education, 
Political Science, and Nursing were under-represented in the combined sample. 
Journalism, Psychology, and Real Estate and Urban Affairs were over-represented 
by more than two percent (see Figure 3-3). 

Figures 3-4a and 3-4b present the SAT scores for the combined sample and 
population. The combined sample scored better on both the Verbal (440.4 for the 
sample and 396.6 xor the population) and Math (452.16 for the sample and 414.4 
for the population). This may indicate a sample that was better prepared in 
high S'-hool than was the population. 

Given these variations in characteristics, students in the combined saniple 
evidenced few differences from those in the population of seniors graduating in 
1986-87 and 1987-88. 



- 63 - 

Si 



Figure 3-1. Distribution of Clayton State Combined San?>le: Gender 

Combined Sample Population 
Gender N PERCENT N PERCENT 



Female 50 65.8% 652 55.4% 

Male 26 34.2% 524 44.6% 

TOTALS 76 100.0% 1176 100.0% 



Figure 3-2. Distribution of Clayton Combined Saa^le: Ethnicity 

Combined Sample Population 
Ethnicity N PERCENT N PERCFJ4T 



Not specified 1 1.32% 3 .26% 

Black 2 2.63% 27 2.30% 

Native American 0 .00% 1 .09% 

Asian 1 1.32% 4 .34% 

Hispanic 2 2.63% 8 .68% 

White/Non-Hispanic 70 92.11% 1,116 94.90% 

Foreign 0 .00% 12 1.02% 

Other 0 .U0% 5 .43% 

TOTALS 76 100.00% 1176 100.00% 
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Fig^ire 3-4a. Siunnary of SAT Scores and Grades for Clayton Combined Sairqples 



St = ?s « s s s = s = s s; s s s; 35 S£ ss :s S££; s; s :s s s S£ IS s s s =; s;: = := = s s s £ s = ^ :^ ^ 

SAT Part Standard Standard 

Score N Mean Deviation Range Error 



Verbal 76 440.41 83.28 240-630 9.55 

Math lb 452.16 92.90 200-640 10,66 

SAT Total 76 892,57 148.28 543-1230 17,01 
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Figure 3-'4b* Summary of SAT Scores and G rades for Population 

SAT Part Standard 
Score N Mean Deviation 



Standard 
Error 



Verbal 189 396.59 

Math 789 414.40 

SAT Total 789 811.48 

Grade Point Average 789 3.00 



74.27 
78.63 
129.27 
.44 



2.37 
2.54 
4.15 
.014 



Figure 3-5. Entering Semester of Combined Sample 



Entering Semester 



PERCENT 



Fall 1958 
Fall 1965 
Fall 1966 
Fall 1967 
Fall 1973 
Fall 1974 
Fall 1976 
Fall 1977 
Fall 1978 
Fall 1979 
Fall 1980 
Fall 1981 
Fall 1982 
Fall 1983 
Fall 1984 
Spring 1985 



1 
1 
2 
1 
2 
1 
6 
1 
1 
3 
6 
4 
16 
17 
12 
2 



1.3% 
1.3% 
2.6% 
1.3% 
2.6% 
1.3% 
7.9% 
1.3% 
1.3% 
3.9% 
7.9% 
5.3% 
21.1% 

15.8% 

2.6% 



TOTALS 



76 



100.0% 
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Figure 3-6. Planned Year of Graduation: Clayton Combined Sait5>le 
Planned Year 

of Graduation N PERCENT 



No Response 19 25.0% 

1986 14 18.4% 

1987 25 32.9% 

1988 14 IB. 4% 

1989 2 2.6% 

1990 1 1.3% 
1992 1 1.3% 
TOTALS 76 100.0% 



Figure 3-5 shows that the overwhelming majority of the combined san^le 
entered the institution in the fall 1982, fall 1983, or the fall 1984 terms. 
All students in the combined san^jle were projected to meet graduation 
requirements for a bachelor's degree either during the 1986-87 or the 1987-88 
academic year (Figure 3-6). 

Students in the combined san^le were clearly planning some form of 
post -baccalaureate study (Figure 3-7). Over one-half (56.6%) planned to pursue 
a master's degree, while nearly one-sixth (15.8%) planned to enter a doctoral 
program. Only 10.5 percent had no plans for subsequent graduate study. No 
comparable information was available for the population of graduating seniors. 

The educational attainment of parents has been shown to be positively 
correlated to student achievement in college. Nearly one-third (32.9%) of the 
fathers and over one-half (56.6%) of the mothers of students had not attained a 
high school diploma. Nearly one-sixth (13.2%) of fathers and one-tenth (9.2%) 
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of mothers had attained at least an associate degree (Figure 3-8). 



Figure 3-7, Degree Objecti'^'es for Clayton CCTxbined San5>le 

Degree Objectives N PERCENT 



No Response 


9 
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Non-degree Study 
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ID 
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Master's Degree 
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.6% 


Intermediate Degree (e.g.. Specialist) 
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3 


.9% 


Doctorate (Ph.D., Ed,D.) 


12 


15 


.8% 


Postdoctoral Study 


1 


1 


.3% 


TOTALS 


76 


100 


.0% 
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Figure 3-8. Educational Attainment of Parents for Clayton CcOTbined SaB5>le 



Highest Level of 
Education Coonpleted 


N 


£ s s :s s; s s£ sss= s£S 

Father 
Percent 


Mother 
N Percent 


No Response 


6 


7.9% 


6 


7.9% 


Grade School or Less 


10 


13.2% 


14 


18.4% 


Some High School 


15 


19.7% 


29 


38.2% 


High School Diploma or Equivalent 


19 


25.0% 


12 


15.8% 


Business or Trade School 


9 


11.8% 


3 


3.9% 


Some College 


7 


9.2% 


5 


6.6% 


Associate Degree 
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7.9% 


3 


3.9% 


Bachelor's Degree 


1 


1.3% 


1 


1.3% 


Some Graduate/Professional School 


1 


1.3% 


2 


2.6% 


Graduate/Professional Degree 


2 


2.6% 


1 


1.3% 


TOTALS 


76 


100.0% 


76 


100.0% 


Figure 3-9. Extent of Community Service Activities for Corabined Sample 


Hours/Week in Community Service 




N PERCENT 


No response 






6 


7.9% 


0 hours 






28 


36.8% 


1-5 hours 






32 


42.1% 


6-10 hours 






6 


7.9% 


11-20 hours 






1 


1.3% 


More than 20 hours 






3 


3.9% 


TOTALS 






76 


100.0% 
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Figure 3-10. In5)ortant Honors and Awards for Coiribined San^Jle 

Type of Honor/Award N Percent 



No response 


6 


7.9% 


Student government or organization 


4 


5.3% 


Professional (an award or prize for field work or 
publication of a scholarly article or book) 


1 


1.3% 


Canmunity service (election or appointment to a 
comrounity service unit, activity, or group) 


7 


9.2% 


Literary (editing the college paper, yearbook, or 
literary magazine or having a poem, story, or 
article published in a public paper or magazine) 


4 


5.3% 


Artistic (a high rating in a music contest, a part 
in a play, opera, or show, or an award in an 
art competition) 


a 

D 




Athletics (a letter in athletics) 


5 


6.6% 


None of the above categories 


41 


53.9% 


TOTALS 


76 


100.0% 



Over one-half (55.3%) of the combined sample students had performed some 
community service during the past year, but for the majority of these students 
this had con^rised less than five hours per week (see Figure 3-9). 

Nearly two- fifths (39.5%) of the sait^Jle students had earned some form of 
professional, community service, literary, artistic, athletics, or student 
government honor, or award. Prior research had shewn such distinctions to he 
highly correlated to student performance, persistence, progress, and degree 
attainment in college. 
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Figure 3-11. Suninarv of Clayton Canbined SaniJle 



76 students 

50 females (65.8%) and 26 males (34.2%) 
70 out of 76 are white (92.1%) 

4 or more students majored in each of Accounting, Journalism, 
Marketing, and Psychology 

16 students (21.1) for B.S. 
15 students (19.7%) for B.A. 
23 students (30.3%) for B.B.A. 

1 student (1.3%) for Bachelor of Music 

8 students (10.5%) for Bachelor of Science in Education 
8 students (10.5%) for P.B. 

2 students (2.6%) for Bachelor of Fine Arts 
1 student (1.3%) for Master of Education 

1 student (1.3%) for Associiate of Science 
1 student (1.3%) for N.D. 



Figure 3-11 summarizes the characteristics of Clayton State College combine 
saii^le group. These students were largely analogous to the population with 
regard to major and race. There was an over-representation of women (10.4%) and 
some variation in the representation of majors in the Saa^jle. The precollege 
achievement variables also showed some discrepency from the population. 



San^le Size: 
Sex: 
Race: 
Major Area: 

Degree: 
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IV. I>elerm.ining Student Learned Abilities 

GKE residual scores 

To control for the effects of the incoming ability of students, the 

predictive effect of SAT scores were partialled from GRE item- type scores. For 

this, 9 GRE item-type residual scores were developed as follows: 

GRE Verbal item-type residuals; 

ANA: Analogies 18 questions 

SC: Sentence Cc^i^letion 14 questions 

RD: Reading Comprehension 22 questions 

ANT: Antonyms 22 questions 

GRE Quantitative item- type residuals; 

QC: Quantitative COT?>arison 30 questions 

RM: Regular Mathematics 17 questions 

DI: Data Interpretation 10 questions 

CRE Analytical item-type residuals; 

ARE: Analytical Reasoning 38 questions 

LR: Logical Reasoning 12 questions 

Each of the 4 GRE Verbal item-type scores were regressed on the SAT Verbal 

scores. Each of the 3 GRE quantitative item- type scores were regressed on the 

SAT mathematics scores. Each of the 2 GRE analytical item- type scores were 

regressed on the SAT total scores. These GRE item-type residual scores were 

referred to as student residual scores, that is, the in^rovement students showed 

in general learned abilities from the time they entered college to the time of 

GRE testing during their senior year. 



Reliability and correlation of GRE item-typ es 

Prior to partialling the effects of the students' SAT scores from their GRE 
item-type scores, the reliability of the GRE item- types for this san^le was 
tested. Next, the correlation between the GRE item-types and the SAT sub-scores 
and total score was examined. Finally, a regression of GRE item-types on SAT 
sub-scores was conducted to calculate student residual scores for each GRE 
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item- type. 

A preljjninary question in the analysis of GRE item-types and sub-tests is 
their reliability within the san^Jle group. Three factors typically contribute 
to the reliability or unreliability of test scores (Ebel, 1972). The first 
factor is the appropriateness and aef initiveness of the questions. On one hand, 
the appropriateness of the questions is presumed by the widespread acceptance of 
the GRE as an examination used in graduate school admissions. On the other 
hand, the appropriateness of the items and item-types may be questioned relative 
to the goals of the general education curriculum of the institution. In this 
sense, the reliability of the GRE may vary from institution to institution. 

A second factor contributing to the reliability of test scores is the 
consistency and objectivity of the person (or in this case, machine) who scores 
the examinations. All the test responses are read by an optimal scanner and 
scored by a coirputer at Educational Testing Service. The accuracy of this 
equipment relative to the task was presumed and not tested. 

A third factor contributing to the reliability is the constancy or 
stability of a student's ability to perform the tasks presented in the test. 
Students may vary from hour to hour or from day to day in their alertness, 
energy and recall; these may affect test performance, reducing the reliability 
of the scores. According to procedures established by the Educational Testing 
Service for the administration of the Graduate Record Examination, all students 
were tested on the first Saturday morning (8:00 a.m. to 12;00 noon) in February 
of each test year. 

Reliability is not merely the properthe GRE itself bat rather of the 
individual item- types relative to the student group examined. The more 
appropriate the test is to the group of students, the higher the reliability, of 
the scores. Ideally, the reliablity of a set of scores may be determined using 
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the correlation coefficient between that set of scores amd another set from an 
equivalent test of the members of the S2une group. In many testing situations, 
including the ones described in this report, a test-retest method of determining 
the reliability of GRE item- types was not available. 

The Guttman Split-Half method of determining reliability estijnates 
reliability by splitting the sang3le into halves and determining the correlation 
between the scores in the two groups. The results of the split-half method are 
dependent upon the manner in which the group is halved. Cronbach's alpha is a 
statistic designed to overcome this problem. It is a generalized formula 
representing the average correlation obtained from all possible split-half 
reliability estimates. 

The results of the reliability analysis for Clayton State San^le «1 is 
presented in Figure 4-la; the reliability analysis for Clayton State Sanple #2 
is displayed in Figure 4- lb. Since different forms of the GRE were used each 
year, the reliability results are presented for each individual sample. For the 
purposes of this study, reliability coefficients at or above a= .65 were deemed 
satisfactory (Mehrens & Lehmann, 1969). Due to the exploratory nature of this 
research, lower reliability coefficients were accepted. In Saii5>le Group #1, 
Logical Reasoning (a= .51) evidenced low reliability. In Sample Group #2, 
Antonyms (a= .54), Analogies (a= .63), Reading Coit^srehension (a= .58), Antonyms 
(a= .54), Quantitative Con^arisons (a= .63), Logical Reasoning (a= .58), Data 
Interpretation (a = .64), and Regular Mathematics (a= .40) showed low 
reliability. In both saii^les, the reliability of the individual item-types 
tended to increase with the number of items ccsnprising the given item- type. In 
the cluster analytic model, the SAT sub-scores are used as measures of entering 
student ability. Prior to regressing GRE item- type scores on SAT scores, it is 
important to determine the extent to which GRE item-types and SAT sub-scores are 
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correlated. For exanyle, determining whether the GRE item- type, Analogies, has 
a stronger correlation with SAT Verbal, SAT Math or the total SAT scores will 
help determine which SAT score should be used in the subsequent regression 
analysis. 



Figure 4-la. Keliability of Coefficients of GRE Item-Types— Sansple #1 
GRE Item- types 



Code 



Number 
of items 



Alpha 


uuutxnan s 
Split-hali 


.6491 


.7563 


.6662 


.4118 


.7090 


.7821 


.8504 


.7941 


.8373 


.8598 


.7806 


.7360 


.6829 


.6155 


.7958 


.6535 


.5074 


.5577 


.9125 


.8919 


.9040 


.8964 


.8018 


.7299 



Analogy 


ANA 


18 


Sentence Coirpletion 


SC 


14 


Reading Comprehension 


RD 


22 


Antonyms 


ANT 


22 


Quantitative Con^ arisen 


QC 


30 


Regular Mathematics 


RM 


20 


Data Interpretation 


Dl 


10 


Analytical Reasoning 


ARE 


38 


Logical Reasoning 


LR 


12 


GRE Verbal 


GRE-V 


76 


GRE Quantitative 


GRE-Q 


60 


GRE Analytic 


GRE-A 


50 
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Figure 4-lb. Reliability of Coefficients of GRE Item-Types— Sample #2 



GRE Item-types 


Code 


Nvuhber 


Cronbach' s 


Guttman' s 




of items 


Alpha 


Split-half 


Anaxogy 




18 


.6347 


.6933 


Sentence Completion 


sc 


14 


.6789 


.4351 


Reading Coraprehension 


RD 


22 


.5787 


.3794 


HnLonyiuS 




22 


.5445 


.5005 


Quantitative Coniparison 


QC 


30 


.5757 


.7915 


Regular Mathematics 


Kn 






5044 


Data xnverpreuaLjLon 






.6400 


.2974 


Analytical Reasoning 


ARE 


38 


.7974 


.7449 


liogical Reasoning 


LR 


12 


.6288 


.5447 


GRE Verbal 


GRE-V 


76 


.8271 


.9086 


GRE Quantitative 


GRE-Q 


60 


.7526 


.7791 


GRE Analytic 


GRE-A 


50 


.8432 


.7674 



Figure 4-2 indicates strong, positive relationships between GRE item-types 
and SAT scores. For the Clayton State Sangile, GRE Verbal item- types were 
strongly correlated to the SAT Verbal sub-score with r ranging from .53 to .63. 
GRE Quantitative item-types had strong correlations with the SAT Mathematics 
sub-score, r ranging from .37 to .60. GRE Analytic item-types evidenced strong 
correlations with the SAT Total score (r = .45 and .53). Results of the 
correlation analysis were comparable to those found in the other institutional 
samples previously analyzed (Ithaca, Mills, and Stanford Sangsle Groups). 
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Figure 4-2. Correlation of GRE Item-Types & SAT Scores— Clayton State Sample 



GRE Item-^types 


Code 


SAT 
Verbal 




SAT 
Math 




SAT 
Total 




Analogy 


ANA 


. boo? 












Sentence Coinpletion 


SC 


.5503 




.2037 




.4367 




Heading Comprehension 


RD 


.6267 




.4061 


3 


.6064 




kntonyms 


ANT 














Quantitative Coinparison 


QC 


.1343 




.6024 




.4528 


4» 


Regular Mathematics 


RM 


.23b0 




cc t c 

. Dblo 








Data Interpretation 


DI 


. 3U(JD 


a 


• ^ / XD 


3 






Analytical Reasoning 




.3834 




.4963 




5263 




Logical Reasoning 


LR 


.4348 




.3248 




.4477 




GRE Verbal 


GR'-V 


.6760 




.3210 


3 


.5808 


i% 


GRE Quantitative 


GRE-Q 


.2353 




.6395 




.5328 




GRE Analytic 


GRE~A 


.4575 




.5154 




.5799 





Minimum 
Maximani 
Mean 

< .05 ''p < .001 
^p < .01 ''p < .0001 



Intercorrelation of GRE item-types 

The internal validity of GRE item-types can be measured by con^aring the 
intercorrelation coefficients of GRE item-types. In the Clayton State Sample, 
the intercorrelations between GRE Quantitative item-types were relatively 
stronger than those between other GRE item-type scores. Each GRE subscore 
tended to have higher correlations with the GRE item-types constructing the 
subscore than with GRE item- types constructing other test subscores. The 
analysis of correlations among GRE item-types shows that the item-types have 
strong internal validity. 

Wilson (1985) has suggested that GRE (and SAT) item-types may measure 
discrete forms of general education abilities. This assertion served as the 
theoretical underpinning for the use and treatment of GRE item-types as 
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discrete, multiple measures of general learning. To test Wilson's assertion, 
the intercorrelation among item- type scores was further examined (see Figure 
4-3). 

In the Clayton State Sample, intercorrelations for Verbal item-types ranged 
from r = .54 (ANT/ANA) to r = .65 (ANT/SC). Intercorrelations for Quantitative 
item-types ranged from r = .47 (RM/DI) to r = .60 (RM/QC). Intercorrelations 
between Analytic item- types were r = .40 (AM/LR). However, Analytic Reasoning 
correlated strongly with Quantitative item-types ranging from .51 (QC) to .56 
(RM). The intercorrelational analyses showed tliat in most instances, less than 
50 percent of the variance in one item-type was explained by that of another. 



Figure 4-3. Intercorrelaticai of ORE Itaai-Types for Clayton State Smfilo 



eSE Iteir-Types 


Code 




SC 




RD 




m 


QC 




RM 


DI ARE 


toalogies 




1.0000 




















Seoteoce CoqpletioD 


sc 


.5717 * 


l.OOO) 


















Readlsg CoqsrehensioD 


RD 


.5698 * 


.6125 




l.OOOO 














totonyns 


m 


.5407 " 


.6474 




.6331 


4 


1.0000 










Quantitative Co(q>arisoD5 


QC 


.2351 ' 


.2707 


\ 


.3908 


:« 


.1372 


1.0000 








Regular Hathemtics 


RH 


.2234 


.2653 


1 


.4081 




.2128 


.6048 




1.0000 




Data Interpretation 


DI 


.2464 ' 


.4350 




.5175 


4 


.2573 * 


.5228 




.4714 "* 


1.0000 


AoalTtic Beasoning 




.2557 ' 


,2675 


\ 


.4923 




.2903 ' 


.5i;i 




.5580 * 


,5575 * 1.0000 


logical Reasnmisg 


LR 


.3998 * 


.5329 




.6831 




.4323 " 


.4297 


<« 


.2987 ' 


.5575 " .3960 



* p < .05 ' p < .001 

' p < .01 " p < .0001 



As Figure 4-4 demonstrates, Clayton State students performed well on the 
GRE General Examination. Students answered the questions correctly in 
approximately 93 of the 18C GRE items. Students attained perfect scores on 
Sentence Completion and Logical Reasoning item-types. 
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Vftiile GRE raw scores were generally and consistently high amon'i these 
students, differences among scores appeared when the effect of the precollege 
learning (as measured by the SAT) was removed. When the theoretical scores (as 
predicted by corresponding SAT scores) were conipared with the students' actual 
responses (Figure 4-5), students showed the largest inqprovement on the Data 
Interpretation item-type and the lowest amount of in?>roved performance on the 
Quantitative Coit^jarisons item- type. 

The greatest amount of variance in item-type residuals, including the 
greatest standard error and standard deviation, were found in the Analytic 
Reasoning and Quantitative Cortqparisons it^-types. The variance in these 
residuals holds implication for the ensuing cluster analysis in that GRE 
item-types with greater variance will play a more significant role in sorting 
courses into clusters. As was discovered in the analysis of saii5>les from other 
participating institutions, those GRE item- types with smaller variance play less 
of a role in discriminating course clusters. 

As Figure 4-5 demonstrates, from one-tenth (Data Interpretation) to 
two-fifths (Reading Comprehension) of GRE item-type score variation among the 
Clayton State Sample was explained by their SAT scores. All regression 
functions were statistically significant at .0001 with the exception of Data 
Interpretation which was significant at .001. Also, the range of residual 
scores did vary considerably across GRE item- types. 

Using the student residuals obtained from the regression analysis above, 
the mean residuals for each course enrolling 5 or more students were calculated 
for all the 9 GRE item- types. Such a procedure does not assume that the 
specific gains of the students enrolled in each course were directly caused by 
that course. Rather, the reBiduals of each student are attributed to all the 
courses in which they enrolled, and the mean residuals for each course serve as 



a proxy measure of student gains. Once courses are clustered by these gains, 
then hypotheses can be generated and tested as to why students who enrolled in a 
given pattern of courses experienced significant gains on one or more of the 
outcomes criteria (i.e., the item-type residuals). 



Figure 4-4. The Distribution of GRE Scores for Students in Clayton State Sample 



= s = = 3 s 3 ss ss ss as s ss s 

GRE Item-types 


Number 
of Items 


Miniinum 
Right 


Maximum Score 
Bight Range 


dampxe 
Mean 


Standard 
Deviation 


Analogy 


18 


0 


16 


16 






Sentence Completion 


14 


0 


14 


14 


8.08 


2.8084 


Reading Comprehension 


22 


0 


21 


21 


10.91 


3.7173 


Antonyms 




0 


20 


20 


10,34 




Quantitative Comparison 


30 


0 


29 


29 


17 .30 


5. 3342 


Regular Mathematics 


20 


0 


18 


18 


9 .20 


3 . 3/84 


Data Interpretation 


10 


0 


9 


9 


4.68 


2.2581 


Analytical Reasoning 


38 


0 


29 


29 


16.72 


5.8506 


Logical Reasoning 


12 


0 


12 


12 


6.28 


2.2838 


GRE Verbal 


76 


0 


66 


66 


39.12 


11.3160 


GRE Quantitative 


60 


0 


53 


53 


31.18 


9.3033 


GRE Analytic 


50 


0 


39 


39 


23.00 


7.0730 


GRE Verbal (converted) 










452.00 


94.6830 


GRE Quantitative (converted) 








458.40 


103.9366 


GRE Analytic (converted) 










481.47 


102.6104 


Minimum 


10 


0 


9 


9 


4.68 


2.26 


Maximum 


38 


0 


29 


29 


17.30 


5.85 


Mean 


21 


0 


19 


19 


10.44 


3.74 


Total 


186 








93.30 


32.58 
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Figure 4-5. Suiranary of Regression Analysis of GRE Scores --Clayton State San?)ie 
Dependent Variables 



Clayton State Sairple 
Ih Students 



GRE Item-types on 



Standard Ad j usted 



SAT Sub- scores 


Code 


F Value 


Deviation 


R-Squa] 


Analogies 


ANA 






£. • O J4 / 




Sentence Conqpletion 


sc 


32. 


148 


2.8084 


.2934 


Reading Comprehenssion 


RD 


4 / • 


AAA 


3.7173 


.3845 


Antonyms 


ANT 


28. 


616 


4.3191 


.2691 


Quantitative Cocoparisons 


QC 


42. 


137 


5.3342 


.3542 


Regular Mathematics 


m 


34. 


089 


3.3784 


.3061 


Data Interpretation 


DI 


11. 


847 


2.2581 


.1264 


Analytic Reasoning 


ARE 


28. 


346 


5.8506 


.2672 


Logical Reasoning 


LR 


18. 


551 


2.2838 


.1896 


Verbal (raw) 




62. 


267 


11.3160 


.4496 


Quantitative (raw) 




51. 


195 


9.3033 


.4009 


Analytical (raw) 




37. 


490 


7.0729 


.3273 


p > F = .0001 

























Re gression analysis of SAT scores on GRE item-type scores 

To determine the extent to which these students showed ijiprovenient over 
their precollege SAT scores, the GRE raw scores were regressed on the 
corresponding SAT scores. GRE Verbal item-types were regressed on SAT Verbal 
sub-scores, GRE yuantitative itein-types were regressed on SAT Math scores, and 
GRE Analytic item- types were regressed on SAT Total scores. The resulting GRE 
item- type correlations with corresponding SAT scores were noticeably lower than 
their corresponding GRE sub-scores. This suggests that individual item-types 
also may measure discrete abilities apart from those of the SAT sub-scores 
and/or they may reflect lower reliability stemming from the fact that there are 
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fewer items coinprising an item-type than a sub-score. 

The SAT scores explained smaller portions of variance in GRE item-type 
scores than in the GRE sub-scores (Verbal, Quantitative and Analytical --see 
Figure 4-5). The SAT Verbal e3q>lained 29.34 percent of the variance in the 
Sentence Con5>letion item-type among the Clayton State SaHi>le. The SAT Verbal 
explained 31.22 percent of the variation in the Analogies item- type, 38.45 
percent of the variation in Reading Catt5>rehension and 26.91 percent in Antonyms 
items among tr.e Clayton State San^^le. The SAT Math scores explained 35.42 
percent of variation in Quantitative Comparison item responses, 30.61 percent of 
variation in Regular Math item-type scores, and 12.64 percent of variation in 
Data Interpretation for the Clayton State SanjJle. The combined SAT Verbal and 
SAT Math scores (referred to as SAT Total) explained 26.72 percent of variance 
in Analytic Reasoning and 18.96 percent of variance in Logical Reasoning for the 
Clayton State Sample. In most instances, the regression model proved 
significant at the .0001 level, suggesting effective control measures for the 
general learned abilities of students as they entered college as freshmen. The 
only exception was Data Interpretation which was significant at the .001 level. 

Clayton State San5>le students entered college with slightly higher mean SAT 
Math score (452) than SAT Verbal score (440). As these Clayton State students 
approached graduation, they remained normatively stronger in Quantitative 
abilities (mean GR£-Q = 458) than in Verbal abilities (mean GRE-V = 452). Yet, 
the regression analysis showed the Clayton State students evinced large variance 
in residuals on specific GRE item-types in excess of those represented in the 
sub-scores of the two tests. For Clayton State, specific and significant 
residual variance was demonstrated in Quantitative Con^jarison and Analytic 
Reasoning . 
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Figure 4-5 coni|5ares the explained variance (R-squared) for each GRE 
item-type, raw GRE sub-score and converted GRE sub-score. Only SAT sub-scores 
were available to the research team; these scores are converted scores . The 
actual scores of a student on a particular form of the SAT test is transformed 
relative to test norms so as to be con5>arable with other forms of the test and 
with other students tested. A similar process is used with the GRE exams. Raw 
GRE sub- scores for a given form of the test are transformed so as to allow 
comparisons with national norms and with scores on other forms of the test. In 
all cases within the Clayton State Sample, the SAT accounted for more variance 
in GRE sub-scores than in the GRE item-type scores. Figure 4-6 illustrates the 
extent of unexplained variance {that is, the variance in GRE item-type scores 
attributable to sources other than the precollege abilities of the students, as 
measured by the SAT), Only the converted GRE sub-scores are graphed in Figure 
4-6. The findings tend to agree with previous reports of this project and those 
by Wilson (1985) suggesting that GRE item-types may have greater correlation 
with variance in learning during the college years than do the GRE sub-scores. 

As has been previously discussed, critics of the GRE and SAT as measures of 
general learned abilities attac)t the validity of the measure themselves. These 
criticisms are based primarily on the use of the test sub-scores and the total 
test scores; the use of the item- type scores on either the GRE or SAT as 
multiple measures of general learning have not been widely explored (Adelman, 
1988). The reliability of GRE item-types, their strong correlation with SAT 
sub-scores, and their apparent ability to measure discrete types of learning 
suggest that they may be of potential value as criteria in the assessment of 
general learning abilities of undergraduates. 

For the purposes of this research, precollege measures of student ability 
were defined as the SAT-V, SAT-M, and the total SAT score. Although there is 
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research to suggest that SAT item- types may have greater predictive validity of 
college performance (Ramist, 1981a; 1981b; Schrader, 1984), SAT item-types were 
not available for use in the research model. Postcollege measures of student 
learning were defined as the 9 item- type scores on the GRE. According to 
Astin's model {1970a, 1970b), the effect of the learned abilities of students 
entering college on student outccsne measures should first be determined. Once 
the variance in GR£ item-type scores attributable to SAT scores was determined, 
the unexplained variance (score residuals) could be used as a proxy of change in 
general student learning along the 9 item-type measures. 

The SAT Verbal scores were used to predict each of the GRE Verbal 
item-types using the general lineal model. The SAT Math scores were used to 
predict each of the corresponding GRE Quantitative it«n-types. The combined SAT 
scores (SAT Total) were used to predict the GRE Analytic Scores. The regression 
analysis summarized in Figure 4-5 was performed by the PROC REG in ^he SAS 
statistical package. Individual student score residuals theoretically ranged 
from +1.00 to -1.00. Since individual scores were predicted relative to the 
sample group, some students will have higher actual scores than predicted 
scores^ while others will have lower actual scores than predicted scores. A 
negative residual represents the unexplained variance of a student whose act\ial 
GRE item- type score was less than that predicted by the student's corresponding 
SAT subscore. Larger unexplained variance on a given item-^type indicated letrger 
amounts of change (either gain or loss) in general learned abilities associated 
with that measure. 

Variation in GRE item-type scores may be attributed to two sources: 
variation due to changes in the independent SAT variable and changes 
attributable to other sources. In most cases in the Clayton State Sample, the 
probability of obtaining the F value was .0001^ suggesting that the general 
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linear inodel is adequate in explaining the sources of variation within the GRE 
item-type scores. R-squared represented the percent of total GRE itero-type 
variation explained by the independent SAT variable. The residual variation in 
each GRE itein-type--that not measured by the SAT 8core--was used as the proxy 
measure of general learned abilities during the undergraduate ejqperience. These 
student residual scores, from the time they took the SAT prior to college to the 
time they took the GRE as graduating seniors on each of the 9 item-types, served 
as the measures of general learning among the Clayton State Sangsle (Hanson, 
1988; Pascarella, 1987). 

Fig 4-6. Chansre ba GRE i ten-tapes 
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V. Review of Literature on Catalogues 
The purpose of the Clayton State and GSU Catalogue studies was to describe 
the intended curriculum in general education for undergraduates. By examining 
the nature, scope, and structure of the curriculum, constraints to students' 
course- taking behavior may be identified. Furthermore, the catalogue studies 
provided a con5)arative basis for examining what the enrollment patterns of 
students who showed gains in general learning were in relation to what the 
college intended the curricular patterns in general learning to be. Sue. m 
examination was admittedly a con5>arison of local reguirements (general education 
degree requirements) with national norms of general learning as measured by the 
GRE examination. 

The Clayton State and GSU Catalogue studies began with a review of 
pertinent prior research. The issues and procedures idei.tif ied in previous 
studies guided the inquiry. The review of selected research in which college 
catalogs served as sources of curricular data and analysis revealed three basic 
avenues of investigation. The first used catalogs to analyze the development of 
a specific academic area or topic. The second avenue of research en^loying 
catalogs studied change in the general education curriculum which occurred over 
a specific time period. The third research area involved the use of catalogs to 
develop course classification systems. 

Catalog studies of specific academic areas 

Becan-McBride (1980), Boysen (1979), Fosdick (1984), LoGuidice (1980), 
and Tenopir (1985) used catalogs to study the status of particular academic 
areas. Two other studies (Gillespie & Cameron, 1986; Grave, 1985) used catalogs 
to analyze the development of a specific academic discipline over time. 
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Boysen (1979) analyEed the undergraduate technical cooinunications 
curriculum at selected colleges of engineering. Institutions were chosen for 
inclusion in the study based on the clarity of written course and program 
descriptions found in the college catalogs. Boysen' s study showed that many 
catalogs are replete with aidbiguities and that generalizations of c\xrricular 
cofliparability may be drawn only to those institutions with clearly written 
catalogs . 

Becan**McBride (1980) used the catalogs and brochures from medical 
technology programs to study characteristics such as course requirements, nature 
of the program, degree earned^ and accreditation. The study found that 
different types of medical technology programs mandated different prerequisites, 
thereby impacting on course- taking behavior evidenced on student transcripts. 

LoGuidice (1983) analyzed the catalogs of Lutheran-affiliated colleges and 
universities :o determine the presence of a global/ international curricular 
perspective. Course titles and course descriptions were examined for certain 
key words and phrases selected a priori . LoGuidice found descriptive 
commonalities across like departments or disciplines, with some phrases 
appearing more frequently in certain programnaatic areas. 

The purpose of Fosdick^s research (1984) was to determine the educational 
impact of information science on library science curricula. Catalog course 
titles and descriptions were used as data sources because Fosdick believed them 
to be more objective than verbal or written questionnaires. He found, however, 
several problems seemingly inherent in catalog research: "... investigation of 
the catalogs does involve the subjective judgments of the surveyor, and a few 
schools do not provide adequate information in their catalogs on which to base 
an appraisal (p, 293). 
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Tenopir (1985) studied the kinds of college courses used to determine a 
specialization in information science through an analysis of course 
descriptions, course titles, and/or departiuent titles. Based on this 
descriptive information, courses were assigned to one classification category. 
Originally selected to provide an unobtrusive means of data collection, Tenopir 
found that the lack of coninon nomenclature in college catalogs made the study 
difficult and time consuming. li«hile many of the same courses existed within all 
parent departments, roost did not include the word "information" in their title 
making initial departmental identification problematic. 

Grace's study (1985) examined the context of higher education 
administration curricula in order to determine the extent of prof ess ionalizat ion 
within the field. She used college catalogs from 1972-73 and 1982-83 to gather 
and cOTipare information on the total nuntoer of program hours required, and the 
number of recommended courses within and outside higher education. Grace also 
included program handbooks and student transcripts in her data collection in 
order to be able to address the issue of intended curricula versus actual course 
taking behavior. 

Gillespie and Cameron (1986) used college catalogs, textbooks, and national 
convention programs to determine the development of the teaching of acting in 
colleges and universities over a forty-year time period. In order to delineate 
appropriate courses for inclusion in the study, Gillespie and Cameron chose only 
those courses in which the word "acting" or a variation of "acting" appeared in 
the course title or description. 

From these studies, it appears that college catalogs can be useful in 
examining the status of specific academic areas or their development over time. 
In each case, while other sources may have been included, the primary data were 
derived from an analysis of course titles and/or course descriptions found in 
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the college catalogs. Howevar, the studies also revealed several weaknesses 
inherent in the use of catalogs (particularly course titles) as primary data 
sources. The ambiguity of the catalogs themselves as well as the varying 
vernacular intra- and interinstitutionally make data gathering difficult and 
time consuming, and analysis more subjective than it at first appears. These 
issues are not mitigated by the use of a j?riori designations which, while common 
in the reviewed research, often necessitate judgmental categorizations. 
Multiple data sources including student transcripts may therefore be in^rtant 
to future curricular research, especially when attea5>ting to understand the 
intended versus actual curriculum. 



Studies of catalog change over time 

Four studies, Hefferlin (1969), Dressel and DeLisle (1969), Blackburn, 
Armstrong, Conrad, Didham, and McKune (1976), and Toombs, Fairweather, Amey, ai^d 
Chen (1989), used college catalogs to examine broader higher education 
curricular issues. Hefferlin studies curricular change based on expansion and 
rfeform over a five-year period. He used five areas of the catalog as data 
sources: program majors, areas of concentration, requirements within majors, 
degree requirements for graduation, and general curricular regulations. 
Hefferlin developed the Study of Institutional Vitality (SIV) model for 
analyzing curricular reform by calculating the proportion of courses that were 
dropped or noticeably changed in departstents where the number of courses 
increased. Reform was also calculated by the proportion of courses added or 
changed in departments where the number of courses increased, reform was also 
calculated by the proportion of courses added or changed in departments where 
the number of courses had decreased. Hefferlin found that expansi.on occurred 
more frequently in upper division courses than in lower division, and that 
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curricula at state collegas expanded more than at any other institutional type, 
•rhe general education con?>onent remained stable during the study period and the 
proportion of the curricula devoted to the academic major increased. 

The other three studi'ss focused priniarily on the status and changes in 
general education curricula over time. Change was basically determined on the 
basis of breadth and depth, breadth being defined as the percentage of course 
requirements represented by general education, and depth as the percentage 
represented by the academic major. 

Dressel and DeLisle (1969) studied 322 college and university catalogs to 
determine change over the ten-year period from 1957-61. They found a decline in 
the specificity of general education requirements, a wide variation in the 
course and credit requirements for academic majors, and an increase in the 
proportion of elective courses permitted in fulfilling degree requirements. 
Part of this latter increase seemed tied to the simultaneous emergence of more 
individualized student learning experiences such as honor programs, advanced 
placement courses, and study abroad programs. 

The study by Blackburn et al. (1976) was, in part, designed to replicate 
the work of Dressel and DeLisle, examining the status of undergraduate education 
between 1967 and 1974. Using a slightly smaller sample (271), this study found 
that the proportion of general education courses required for a degree decreased 
during this timeframe while the proportion of coursework devoted to the major 
changed very little. This resulted in a net gain in the elective course area. 
Using transcript analysis, Blackburn et al. went on to examine the course- taking 
behavior of students during this same time period as corapared to degree 
requirements evidenced in the catalogs. Again, though the sample included in 
the transcript analysis was very small, the study found a high level of 
congruence between stated degree requirements and student course-taking 
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behavior. The researchers concluded that there was a need to examine the 
relationships between different teaching techniques, curricular programs, and 
different learning outcomes. 

Toombs et al. (1989) sought to show change in the general education 
curricula since the Blackburn study. Their research involved an analysis of 
data from 100 1986-87 college and university catalogs including degree, general 
education, and major requirements as well as descriptive information found in 
curricular mission statements. The research showed an increase in the 
proportion of the curriculum devoted to general education since the 1976 study, 
while the total credits for a baccalaureate degree remained stable. The 
increase primarily occurred through an expansion of technical course 
requirements such as writing, speech, conputer and quantitative reasoning; 
little increase was found in humanities, social sciences, and nature sciences. 

Each of the preceeding studies acknowledged the difficulties in using 
catalogs us primary data sources described by Dressel and DeLisle in the 
1960's. Catalogs are often poorly organized, inaccurate, ambiguously written, 
and intra- and interinstitutionally inconsistent. Hefferlin's SIV model, 
Blackburn's transcript analysis, and Toombs' sample size and descriptor analysis 
can all be seen as ways of methodologically iiiproving studies using catalog 
data. The use of transcript analysis, while making the most viable connections 
between theory (catalog data) and practice (evidence of student course-taking 
behavior), also proved to be slow and expensive. 

Studies of course coding and classification 

Two fairly recent studies used college catalogs in the development of 
course classification systems at the state and national levels. Waggaman (1980) 
examined the effect of various forms of credit and non-credit designations on 
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the Florida Statewide Course Number System (SCNS). The significant variation 
that existed made it difficult to administer student credits when transferring 
from one Florida institution to another. Waggaxnon studied catalogs, various 
state and federal laws, and accreditation documents from professional 
organization;* . This analysis ltd to an examination of past and present credit 
practices in Florida as well as reccaanendations for specific policies to reduce 
the existing problems in course classifications by level and department. 

On a national level, the Classification of Secondary School Coxxrses project 
(HCES, 1982) used course descriptions collected from high school catalogs to 
develop a nationwide inventory of secondary school courses. A panel of 
reviewers examined the catalogs and established a course title index for each 
course, a unique 6-digit code number, keyword descriptors, and alternate course 
titles. The director was designed to be used by NCES for coding high school 
transcripts. 

Discussion and results of the literature review 

Several themes emerge frcan this literature review relevant to the use of 
catalogs as primary data sources for studying the curriculum. The studies 
suggest that, when studying curricular change especially theory to practice, it 
is necessary to include multiple data sources. This may simply involve 
utilizing various sections of a college catalog, for example mission statements 
and course descriptions in addition to course titles and degree requiren^nts . 
Increasing the san?5le size or using catalogs from several academic years can be 
useful in studying breadth and depth of curricular issues. Student transcripts 
can add a comparative perspective between intended and actual curricula. A 
careful reading of the catalog must still be included, however, to identify 
prerequisites, nvimber of courses required or recommended within and outside a 
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field of study— all of which effect course-taking behavior but which may not be 
easily differentiated when using transcripts alone. The studies indicate that 
college catalogs^ especially in conjunction with other data, can also be used to 
demonstrate change in academic departments, general education, and the 
proportional relationships of curricular con5>onents at progratDttatic and 
institutional levels. 

From a methodological perspective, common research designs were used in the 
studies discussed above. Generally, these studies used predetermined categories 
of courses established de novo or derived from previous research. Sii!5>le word 
counts or qualitative content analyses of course titles and/ or descriptions were 
used to determine placement within each category, but there was no evidence that 
formal procedures were used to examine the content or concurrent validity of the 
categories themselves. Furthermore, ambiguities and inconsistencies in wording 
made it difficult to assign courses to predetermined categories or to identify 
comparable programs. In response, some researchers erected more discrete and 
exacting operational definitions upon which to judge and classify courses. Even 
with these more stringent atten^Jts, there remain questions as to the validity 
and reliability of classifications which depend on clearly worded course titles 
and descriptions, as well as questions regarding inter- rater reliability in 
categorizing the data. Since a major attraction of catalogs as primary data 
sources has been their objectivity and accessibility, it is important to 
recognize the researcher subjectivity inherent in collecting and analyzing 
catalog information. 
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The Structure and Content of General Education 
at Clayton State College and GS13 

Clayton State College is a sinall, senior college within the University 
System of Georgia, Founded in 1969 as a junior college, it remained Clayton 
Junior College until 1985 when it became a senior college and acquired its 
present name. At the time of this study, Clayton College offered the Bachelor 
of Business Administration and Bachelor of Science in Nursing, as well as 
Associate of Arts degrees and an Associate of Science in Aviation 
Administration. The curriculum includes 5 majors, with 4 of these majors from 
the School of Business. Most of the work done was in preparation for one of 
various transfer programs or one and two year career and technical programs. 
All of the students in this study are transfer students. 
General Education Requirements at Clayton State College 

Clayton State College core corriculum is designed to meet the 
specifications of the University System of Georgia. Students who successfully 
complete the approved core or any portion of it at Clayton are eligible to have 
these courses transfered to any other institution within the University of 
Georgia System. This basic core of general education subjects includes 60 
quarter hours of coursework and is drawn from three customary groupings of 
disciplines: Area I: Humanities, Area II: Mathematics and Natural Sciences, 
and Area III: Social Sciences. All transfer programs conform to this 
distributional pattern of general education requirements that follows: 
Core Area I— HUMANITIES. Analysis of the humanities requirements 
reveals a requirement for courses or cc«i5>etencies in English 111, and 
112 for all transfer programs. The students may then choose from a 
list of courses 10 credit hours to con55lete the requirement for the 
humanities. English courses on this list are 201, 202, 211, 212, 221, 
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ana 222, all of which are literature survey courses. Art courses 
include 211 and 213. The remainder of the qualified courses are Drama 
211, Music 211, Philosophy 211, and Speech 121. It is also possible 
to fill the remainder of the humanities core with a sequence of 
language courses, but both courses must be in the san» language. 
There is no requirement for demonstration of a proficiency level in a 
foreign language. 

Core Area II— HATHSilATICS AMD NATORAL SCTKHCRS. All students in a 
transfer program are required to take 5 credit hours in mathematics. 
This requirement is satisfied by choosing one of the following 
courses: Math 105, 111, 112, 121, or 151. A prevailing pattern of a 
10-hour sequence in a laboratory science tends to accoiJ5>any a general 
natural science requirement. Studen'^is are given the option of taking 
one of the following sequences: Biology 111 and either 112, or 113, 
Chemistry 111, 112, Chemistry 151, 152, Physics 151, 152, Physics 251, 
252. The remaining 5 hours needed to satisfy the Mathematics and 
Natural Science core can be chosen from the following courses: Chem 
121, Computer Science 201 or 210, Sci 105 or 110, or any Math course 
numbered 100 or higher, except 107 or 122. 

Core Area III— SOCIAL SdEHCES. History 251 or 252, Political Science 
111, and History 112 are the social sciences requirements for 15 of 
the 20 hours- The remaining 5 hours can be chosen from the following 
courses: Economics 110, Psychology 201, or Sociology 105. 



Structure and Content of General Education Requirements at G5U 

All the students in the Clayton State San^les began their educational 
program at that institution, then transferred to Georgia State University to 
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-omplete their baccalaureate program. Thus both Clayton State and GSU 
curriculum are germane. Undergraduate coursework at GSU is structured according 
to the institutional mission and by the baccalaureate degrees offered. The 
study of GSU catalogs from 1983-84 to 1987-88 began with an examination of 
institutional mission and undergraduate degree requirements. 

The Carnegie Classification scheme provides a means for placing all 
colleges and universities within a comprehensive categorization system of six 
institutional types ("Carnegie Foundation," 1987). According to this typology, 
Georgia State University is a public doctorate-granting institution. To fulfill 
its threefold mission, it promotes the advancement of knowledge through excel- 
lence in teaching, research, and public service. The institution's mission 
focuses on developing in students the requisites for coo^etence, personal 
fulfillment, and responsible leadership in businiss and the professions, in the 
sciences, in the creative and performing arts, in government and in public 
service. 

Georgia State University has five colleges that offer undergraduate 
degrees: Arts and Sciences, Business Administration, Education, Health Sciences, 
and Public Affairs. The University requires each student seeking the baccalau- 
reate degree to coirqplete satisfactorily a general education con^nent. Included 
in tl.is section are the course numbers listed in the 1986-87 and 1987-88 GSU 
catalogs which constitute the requirements relative to the breadth of the 
curriculum (Blackburn et al. , 1976). The purpose of this part of the catalog 
study was to identify the minimum general education requirements of each degree 
offered by the University. For each degree there are other requirements (listed 
as Areas IV, V, VI, and VII) which specify depth (major concentration) 
requirements which are not listed here. 
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Distribution requirement 

The exact nature of the general education core requirement varies swnewhat 
by college and degree. This basic core of general education subjects includes 
60-80 quarter hours of coursework and is drawn frosn three custcinary groupings of 
disciplines: ftxea I: Humanities, Area II: Mathematics and Natural Sciences, 
and Area III: Social Sciences. All colleges conform to the distributional 
pattern of general education requirements, with minor variations, that follows: 
Core Area I— HUHAHITIES. Analysis of the humanities requirements 
suggests a tendency to require courses or conf^etencies in English 111, 
112, 113, 201, Speech 101, 150, and Philosophy 241. The fine arts are 
also included to represent the practice and trends in the whole field 
of the humanities. Two other humanities options that appear are Art 
History 170, 175, 180, and Music 161, 193/393. 

There is no requirement for demonstration of a proficiency level 
in a foreign language sequence, although the catalog offers classical 
and modern foreign language sequences in Arabic, Dutch, French, 
German, Greek, Hebrew, Italian, Japanese, I^tin, Russian, 
Scandinavian, and Spanish. Reference to the Department of Foreign 
languages accounts for additional courses that may be taken to fulfill 
the 20 quarter hour humanities requirement. 

Core Area II— MATHSHATICS AKD NATUBAL SCIE!iCSS. Only the Bachelor of 
Business Administration and the Bachelor of Social Work specify a 
requirement of mathematics and natural science to be met. A 
prevailing pattern of a 10-hour sequence in a laboratory science tends 
to acconpany a general natural science requirement. The widely 
varying practices exhibited in the choices of Mathematics and Natural 
Sciences courses are represented in selections from Astronomy 101, 
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102, Biology 14i, 142, Chemistry 101, 102, Computer Science, Decision 
Sciences 104, 122, Geography 103, 104, Geology 101, 102, Mathematics 
107, 211, 212, and Physics 101, 102. There is some ambiguity as to 
what is included in the sciences. On this point, geography is 
included among the alternatives available in the social sciences 
distribution requirements. 

Core Area III— SOCIAL SCISIKXS. History 111 and 112 are prominent 
among the social sciences requirements, either as an alternative to or 
an equivalent of History 113. Political Science 101 is a requirement 
in the Bachelor of Social Work degree and Political Science 201 is 
required for *• >e Bachelor of Business Administration. Occasionally 
specific reference is given in the degree requirements to behavioral 
science electives. Among the available options in the social sciences 
distribution requirements, two courses may be selected from the 
following: Anthropology 201, 202, 203, Economics 210, Geography 101, 
205, Philosophy 101, 202, 203, 204, or Sociology 201, 202. 
The general education requirements attach some importance to the level of 
the course. With the exception of the dual listed M13S 193/393 in the GSU 
catalog, no 300 or 400- level courses are listed. In contrast, equal numbers 
(29) of 100~level and 20D-level courses are either reconmended or required for 
degree completion in the GSU catalog. This was not the case for the CSC 
catalog. Listed in the CSC catalog were 33 100-level courses conpared to only 
IB 200~level courses, with the bulk of the 200-level in Core I. Figure 5-1 
reveals the increased availability of 100-level courses, reflecting nearly the 
same percentage of course level ^aken as that available. Furthermore, 
requirements in Core Areii II are primarily fulfilled through freshman level 100 
courses for both GSU and CSC. In contrast, slightly less than two-thirds of the 
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courses identified for Core Area III, Social Sciences, requirements are 
sophomore level in the GSU catalog and nearly one-half in the CSC catalog. 
There is an even distribution of Core Area I, Humanities, requirements between 
freshman and sophomore level courses in the GSU catalog. The CSC catalog showed 
nearly two-thirds cf the recomnended courses are 200-level. Courses intended to 
develop students' quantitative abilities are clearly lodged in the freshman year 
experience, while courses intended to develop verbal abilities are distributed 
evenly across the lower division of the curriculum. No junior or senior level 
course is specifically identified as contributing to the general education off 
GSU/CSC students. 

Figure 5-1. Distribution of core areas by level of CSC/GSU course. 













Freshmen 


Sophomore 


Total 




100 Level 


200 Level 


Courses 


Core Area I. 


126 


80 


206 


Humanities 


61.2% 


38.8% 


100.0% 


Core Area 11. 


198 


29 


227 


Mathematics & Natural Sciences 


87.2% 


12.8% 


100.0% 


Core Area III. 


216 


138 


354 


Social Sciences 


61.0% 


39.0% 


100.0% 


Total 


540 


247 


787 




68.6% 


31.4% 


100.0% 











Variation and Change in the GSU and CSC Catalogs 
The final step in the CSC and GSU study involved looking for variance in 
the curriculum by examining catalogs from the 1983-84 academic year through the 
1987-88 academic year. The courses selected to represent the curriculum during 
this period were those unduplicated courses taken by the students in the Clayton 
State College Combined San^jle. The total number of courses examined was 743. 
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For each course a determination was nvade as to whether the course appeared in 
each catalog within the time frame indicated and a notation was made. Courses 
dropped from the curriculum and departmental code chamges were also noted. 
Sin^j > frequencies of these changes were then tabulated. 

During the time indicated the Departir»nt code initials for four departments 
changed in the GSU catalog while only one changed in the CSC catalog. In the 
1982-83 GSU catalog Drama (DRAM) courses became Theater (TH) or Film 
(FILM) courses. Information Systems (IS) courses became Conputer Information 
Systems (CIS) courses in the 1984/85 GSU catalog, and BOED became OADT in the 
same year for CSC. Also, it was noted that in the 1986-87 GSU catalog the Art 
courses were listed under sub- specialties of Art such as drawing, crafts, 
sculpture, etc. An exanrple would be that a drawing and/or painting course now 
has a listing of DP rather than ART, even though the course is still offered 
through the Art Department. In the same catalog the departmental codes for the 
college of Education also underwent a total revision. When these changes 
occurred, it appeared that there was also an increase in the number of courses 
offered in the revised areas. 

The CSC catalog showed remarkable stability during the time span reviewed. 
As figure 5-2a indicates there were alurast no changes made to the courses 
appearing on the trancripts as CSC courses. There were but two changes made 
from 1983 to 1988. 

Within the time frame under investigation, there were ninety-seven course 
title changes noted in the GSU catalogs. These were cases where the course 
lasting and course number remained the same, yet the title was simply altered. 
It cannot be determined from a catalog study whether all title changes reflect 
true change in course content or whether the title was changed to more 
accurately reflect what was being taught in the courses. 
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Ah Figure 5-2b illustrate, most of the curricular change occurred during 
two years ^ 1984'-85 and 1985-86. The total number of course additions and 
deletions represented no greater than 4.74 percent for any one year (1984-85) 
and no inore than 14.3 percent cumulative change since the 1983-84 academic year. 
These findings suggest a smll amount of variability in the curriculum over the 
time period studied. Unlike the conclusions of earlier catalog studies 
(Blackburn et al., 1976; Dressel and DeLisle, 1969; Hefferlin, 1969), the rate 
of curricular change at GSU during the period studied was not rapid. The 
assun^tion that course content and curricular content remained basically stable 
over time appeared to be confirmed. 

Figure 5-2a, Curricular change in CSC College catalogs, 1983-1988. 
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Figure 5-2b. Curricular change in GSU College catalogs, 1983-1988. 
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Figure 5-3a reveals the stability in Clayton State Colleges departnjental 
programs. Only two departjnents deleted a course taken by the san?>le. This was 
not the case for the GSU departJuents as illustrated by figure 5-3b. Certain 
deparcments evidenced 9 high number of curricular additions. For example. Art, 
Computer Science (CSC), Geology (GEOL), Nutrition/Dietetics (NTD) departments 
had substantial additions as well as the most net curricular change. Given 
Hefferlin's hypothesis (1969) that high frequency of curricular change occurs in 
environments of institutional instability and that low frequency of curricular 
change occurs in environments of institutional stability, further research may 
be warranted to determine if the hypothesis is applicable to different levels of 
the organization such as departments. Did faculty and staff in the departments 
with a high degree of net change sense greater departmental instability and 
disarray in the years preceding the catalog changes? Did faculty and staff in 
comparably sized departments with low curricular change experience greater 
levels of departmental stability? Such questions are beyond the scope of the 
current research project, but may warrant investigation by other res, ♦..archers. 
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Figure 5-3a. Curricular change by CSC department or pkDgram 
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Figure 5*3b. Curricular change by GSU department or prograji 
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.5 


1.6% 


Maximum 


3 


12.'?% 


5 


16.1% 
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a. 5% 


8 


25.0% 
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.0% 
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.0% 


0 


.0% 
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-15.6% 
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3.1% 


1,2 


3.8% 


2.3 


2.4% 


2.2 


7.0% 





















Findings and conclusions from the CSC and GSU catalog study 

Like previous studies (Boysen, 1979; Dressel & DeLisle, 1969; Fosdick, 
1984; Tenopir, 1985), the CSC and GSU catalog study encountered difficulty in 
placing courses into the predetermined categories and identifying like programs. 
Often judgments needed to be made on the basis of catalog descriptions as to 
whether a course had undergone substantial revision, was equivalent to a course 
appearing in a previous catalog under a different number and/or department, or 
was cross-listed with a course in another department. It was also ambiguous to 
whether certain courses were reccsnmended or required to meet the general educa- 
tion core requirements of the various degrees at CSC and GSU College. In short, 
ambiguities in wording and phraseology made the catalog einalyses difficult, 
l^esearch on inter-rater reliability in such catalog analyses is needed. 
However, the catalog study was n*»cessary and essential in order to con5>are and 
contrast the similarities and differences between the intended general education 
(as stated in the catalog) with the actual general education of students (as 
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evidenced on their transcript. 

The review of the prior catalog studies identified several methodological 
and substantive issues. First, prerequisites appeared as variables posing 
potentially significant constraints on course-taking behavior (Becan-McBride, 
1980). The CSC and GSU Catalog Study indicated a low percentage of course 
prerequisites had been ignored. This suggested that students were constrained 
in their course-taking behavior by course prerequisites. 

A second issue discussed in the prior catalog studies was that course 
content may exhibit commonalities across institutions by academic department or 
program (Dressel and DeLisle, 1969) and among institutions by type 
(Becan-McBride, 1980; Boysen, 1979; LoGuidice, 1983; Tenopir, 1985). However, 
general education requirements may be subject to broader social and intellectual 
trends and may vary regardless of institutional type (Dressel and DeLisle, 
1969). These observations required inter institutional ccai5>arisons of general 
education requirements and of the content and distribution of courses within 
departments. Such analyses require con^sarable catalog studies of the other 
institutions participating in this project. Since there is but one institution 
per Carnegie classification ("Carnegie classification", 1987; Ratcliff, 1986), 
conqparisons of catalog content within a given type of inst5*-ution were not 
possible within the scope of the current research project. 

Prior literature had suggested that the number of hours required of a 
specific program, the number of reccsnmended and the number of required courses 
within a field of study and outside a field of study were also found to be 
potential constraints on study covirse selection and enrollment (Dressel and 
DeLisle, 1969; Grace, 1985). The general education requirements for the 
baccalaureate degrees offered by the colleges of GSU varied considerably in 
their prescriptivity, suggesting that course enrollment patterns may vary by 
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degree and college. Such variation and its ij!5>act on GRE item-type residual 
scores will be e3q>lored in subsequent reports. 

Specific subjects or departifients were identified in the GSU general 
education requirements for one or more of the bachelors degrees. The courses 
within the Department of Mathematics were generally prescribed or recoosnended to 
meet Core Area II of the GSU general education requirement. The inclusion of 
mathematics courses in coursework patterns found by cluster analysis of student 
transcripts not only tended to confirm the prescription of mathematics courses, 
but also suggested which of those co'irses students who performed well on the GRE 
quantitative item-types enrolled. 

There appeared to be some contradictions within the GSU general education 
requirements. Specifically, PHIL 201 and AH 170 and AH 175 were applicable to 
Humanities (Core Area I) in one instance and to Social Science (Core Area II) in 
another. Appearance of these courses within coursework clusters associated with 
high gains in verbal abilities would affirm the notion that students choosing 
these courses often also perform well on those sections of the GRE traditionally 
associated with the Humanities. Similarly, appearance of these courses in clus- 
ters related to analytic reasoning may suggest relationships with either the 
Humanities or Social Sciences. 

Particular value was placed on laboratory science courses taken in a 
sequence in meeting the GSU general education requirements for Natural Science 
(Core Area II). The appearance of these specific laboratory course sequences in 
the coursework patterns will be examined. 

The CSC and GSU Catalog Study revealed that the intended undergraduate 
general education was primarily (but not exclusively) confined to freshman and 
sophomore (100-200 level) courses. Analysis of the distribution of courses by 
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level appeared warranted for the coursework patterns analysis. 

The above findings and observations frran the GSU Catalog provide a basis 
for inaking jonqparisons between the intended curriculum and the enrollment 
patterns of the GSU Sanple «1. In the following Chapter, the results of the 
quantitative cluster analysis of GSU coursework is presented. The results are 
then contrasted with the findings of the GSU Catalog Study. 
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VI. Quantitative Cluster Analysis of 
Clayton State College Sacrple 

Overview 

This section reports the use of the quantitative cluster analytic procedure 
to analyze the Clayton State san^^le. The findings from the analysis of the 
sanple is presented. The objects of these analyses are the courses which 
constitute the enrollment patterns of students in the Clayton State sainple. The 
demographic profile of these Clayton State students was presented in Section 3. 
In Section 4^ the distribution of GRE and SAT scores was presented. Also in 
that Section^ some basic information on the distribution of courses on the 
students' transcripts was also presented. The subject of the overall research 
is the coursework in which students enrolled, not the students themselves. The 
criterion variables in the research are the GRE item-type residuals • The 
distribution of those residuals among the coursework is described below. 

Clayton State College Samp le 

There were 3,427 courses listed on the 76 transcripts of the students in 
Clayton State sample, indicating that, on average, each of these students had 
enrolled in 45 courses as part of the baccalaureate degree program. Certain 
departments predominated in those enrollment patterns. Looking at the total 
(duplicated) course count of 3,427, 12 departments had their courses appear 100 
times or more on the student transcripts. 
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The 100 or nvore duplicated courses were taken from each of the following 



departments; 




Department 
Code 



Course Count 
(Duplicate) 



Department 



AC 
BIO 
ECON 
ENG 
HIS 
MATH 
MGT 
MUS 

POLI 

PSY 

SOC 



102 
i54 
125 
25& 
185 
227 
112 
131 
121 
110 
208 
110 



Accounting, 

Biology, 

Economics, 

English, 

History, 

Mathematics, 

Management, 

Music, 

Physical Education, 
Political Science, 
Psychology, 
Sociology. 




English was clearly the department of most frequent enrollment. 

Figure 6-1 shows that the lower division (64.80%) leads the upper division 
(35.20%) in enrollment, as demonstrated on the transcripts. 

Figure 6-2 shows that the courses on the transcripts were taken by some 
students over a period of years. Twenty percent of the students enrolled in 
coursework from 19d8 to 1981. These students were non-traditional and likely 
proceeding on a part-time basis. 
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Figure 6-1. Distribution of courses by course nmnbering 

Cumulative 

Course level Pre<iuency Percent Percent 



00-99 

100-199 

200-299 

300-399 

400-499 

500 or higher 

TOTALS 



659 
1,031 
530 
666 
673 
11 
3,570 



19.20% 
30.10% 
15.50% 
19.40% 
19.60% 
.30% 
100.00% 



15.10% 
45.20% 
60.70% 
80.10% 
99.70% 
100.00% 

:oo.oo% 



Figure 6-2. Year and Semester of Enrollment: Clayton State Sainple 



Cumulative 



Year 


Frequency 


Percent 


Percent 


1958 


2 


.10% 


.10% 


1965 


16 


.50% 


.60% 


1966 


9 


.30% 


.90% 


1967 


22 


.60% 


1.50% 


1968 


15 


.40% 


1.90% 


1973 


19 


.60% 


2.50% 


1974 


11 


.30% 


2.80% 


1975 


9 


.30% 


3.10% 


1976 


125 


3.60% 


6.70% 


1977 


38 


1.10% 


7.80% 


1978 


57 


1.70% 


9.50% 


1979 


47 


1.40% 


10.90% 


1980 


152 


4.40% 


15.30% 


1981 


178 


5.20% 


20 . 50% 


1982 


279 


8.10% 


28 . 60% 


1983 


474 


13.80% 


42.40% 


1984 


601 


17.50% 


50.90% 


1985 


525 


15.30% 


75.20% 


1986 


472 


13.80% 


89.00% 


1987 


315 


9.20% 


98.20% 


1988 


61 


1.80% 


100.00% 




3,427 


100.00% 


100.00% 











Total 
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Figure 6-3. The aistribution of GRE item-type resiauals for 1,088 unduplicated 
courses in Clayton State College San^le 

GRE I tern- types Niunber Max Min Score Residual Std Error Std 

of Items Value Value Range Means of Mean Deviation 



Analogy 


18 


4. 


03 


-4. 


22 


8. 


25 


.0718 


.0526 


1.7337 


Sentence Completion 


14 


5. 


11 


-5. 


83 


10. 


94 


.0632 


.0568 


1.8724 


Reading COT^rehension 


22 


5. 


91 


-6. 


78 


12. 


69 


-.0831 


.0706 


2.3292 


Antonyms 


22 


8. 


94 


-9. 


51 


18. 


45 


.0810 


.0792 


2.6134 


Quantitative Ccaiparison 


30 


10. 


56 


-9. 


54 


20. 


10 


-.0934 


.1096 


3.6143 


Regular Matheioatics 


20 


8. 


03 


-4. 


34 


12. 


37 


.2481 


.0747 


2.4632 


Data Interpretation 


10 


4. 


50 


-4. 


93 


9. 


43 


-.9960 


.0530 


1.7475 


Analytical Reasoning 


38 


11. 


02 


-14. 


64 


-10. 


69 


-.6036- 


.1168 


3.8539 


Logical Reasoning 


12 


5. 


22 


-4. 


84 


10. 


07 


-.2715 


.0534 


1.7621 


GRE Item-types: 






















Minimim 


10 


4. 


03 


-14. 


64 


-10. 


69 


-.9960 


.0530 


1.7475 


Maximum 


38 


11. 


02 


-4. 


22 


20. 


10 


.2481 


.1168 


3.8539 


Mean 


21 


7. 


04 


-7. 


18 


10. 


18 


-.2069 


.0768 


2.5320 


Total 


186 














-1.6554 







Figure ^-4. The distribution of GRE item-type residuals for 506 Clayton courses 
used in the Qualitative Cluster Analytic Procedure (Procedure 2). 

GRE Item-types NuiriDer Max Min Score Residual Std Error Std 

of Items Value Value Range Means of Mean Deviation 



Analogy 


18 


4 


.03 


-4. 


06 


8. 


,10 


-.0295 


.0551 


1.2398 


Sentence Con^letion 


14 


4 


.70 


-5. 


83 


10. 


.53 


.0363 


.0632 


1.4212 


Reading Coniprehension 


22 


5 


.06 


-6. 


78 


11. 


.84 


.0143 


.0861 


1.9368 


Antonyms 


22 


7 


.86 


-8. 


41 


16. 


.28 


-.1476 


.0901 


2.0264 


Quantitative Comparison 


30 


8 


.23 


-9. 


54 


17, 


.76 


.1616 


.1334 


3.0015 


Regular Mathematics 


20 


8 


.03 


-4. 


34 


12. 


.37 


.1169 


.0886 


1.9934 


Data Interpretation 


10 


3 


.97 


-4. 


93 


8, 


.90 


-.0675 


.0624 


1.4030 


Analytical Reasoning 


38 


9 


.84 


-10. 


00 


19, 


,84 


-.1819 


.1304 


2.9341 


Logical Reasoning 


12 


4 


.51 


-4. 


84 


9, 




-.1020 


.0559 


1.2568 


GRE Item-types: 






















Minimum 


10 


3 


.97 


-10. 


00 


8 


.10 


-.1819 


.0559 


1.2568 


Maximum 


38 


9 


.84 


-4. 


06 


19 


.84 


.1616 


.1334 


3.0015 


Mean 


21 


6 


.53 


-6. 


53 


12 


.77 


-.0212 


.0888 


1.9967 


Total 


186 














- . 1699 
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Figure 6-5. The distribution of GRE item- type residuals for 177 Clayton courses 
used in the Quantitative Cluster Analytic Procedure (Procedure 1). 

GRE Item-types Number Max Min Score Residual Std Error Std 

of Items Value Value Range Means of Mean Deviation 



Analogy 


18 


3.04 


-2.38 


5 


.42 


.0676 


.0680 


.9046 


Sentence Conipletion 


14 


3.22 


-1.76 


4 


.97 


.1770 


.0643 


.8556 


Reading Comprehension 


22 


4.82 


-3.32 


8 


.14 


.1703 


.1038 


1.3812 


Antonyms 


22 


5.10 


-4.23 


9 


.33 


-.0605 


.1074 


1 . 4289 


Quantitative Coaiparison 


30 


7.39 


-5.27 


12 


.66 


.4531 


.1614 


2.1470 


Regular Mathematics 


20 


8.03 


-2.50 


10 


.53 


.2035 


.1310 


1.7433 


Data Interpretation 


10 


3.97 


-1.94 


5 


.91 


.1861 


.0794 


1.0563 


Analytical Reasoning 


38 


4.23 


-5.66 


9 


.89 


.1174 


.1408 


1.8733 


Logical Reasoning 


12 


2.18 


-1.91 


4 


.08 


.1632 


.0637 


.8477 


GRE I tern- types: 


















Hinijnuin 


10 


2.18 


-5.66 


4 


.08 


-.0605 


.0637 


.8477 


Maximum 


38 


8.03 


-1.76 


12 


.66 


.4531 


.1614 


2.1470 


Mean 


21 


4.67 


-3.22 


7 


.88 


.1763 


.1065 


1.4167 


Total 


186 










1.4102 







Fig 6-6. Cowparison of stAndard deviati 

AU Cr5 
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177 Cr« 



GRE i ten-type residuals 



For each course taken by the Clayton State Sample, the mean of GRE 
item-type residuals for students enrolled in the course was calculated. As 
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means for GR£ residuals for all courses, for courses enrolling 5 or more 
students and for courses enrolling 2 or more students were calculated, certain 
trends emerged. Examination of Figures 6-3 to 6-6 show that as student data 
were aggregated according to courses enrolling 5 or more students, the standard 
deviation of these means was considerably smaller than those for all courses. 
This trend suggested that there are relationships between the coursework taken 
and the student residual scores on the tests. 
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Cluster Analysis of Clavton State Sample 
Creating the raw data matrix and the reswnblance matrix 

Using the mean residuals of the Clayton State Sample and the 177 courses 
found on 5 or more of the Clayton student transcripts, a raw data matrix was 
created. The data matrix consisted of 177 columns and 9 rows (177 x 9). The 
rows represented the criterion variables: the 9 GRE item-type scores. The 
columns represented those courses enrolling 5 or more students. Thus, each cell 
value of the matrix was a man GRE item- type residual score for each course. 

A resemblance matrix was created next to describe how closely each course 
resembles the other 176 courses according to the criterion variables: the 
student residuals. To calculate the resemblance matrix, the correlation 
coefficient was selected as a similarity measure. The correlation coefficient 
was the Pearson product -moment correlation coefficient. Thus, this coefficient 
assesses a pattern similarity of any two courses explained in terms of the 9 GRE 
item-type residuals. 

The resemblance matrix produced in this step consisted of 177 rows and 177 
columns (177 x 177), in which each cell value theoretically ranged from -1.00 to 
1.00. The calculation of the resemblance matrix was done using the SPSSx PROX- 
IMITY program. The program provides 37 different proximity measures. Ten of 
them are for quantitative data and the remainder are for binary or qualitative 
data. This program can directly produce distance, dissimilarity, or similarity 
matrices as text files for a small to moderate number of cases and variables, 
which can be directly used for other SPSSx procedures or for other statistical 
programs. %VD? PIM can also be used to calculate the correlation resemblance 
matrix and save it as a text file. 
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Selection of the clustering method 

The method selected for this analysis was the average linkage method 
(UPGMA)t The appropriateness of the Average Linkage Method was discussed in the 
previous project reports (Ratcliff , 19B8ar 1988b) project. Contrary to reports 
by Romesburg (1984), UPOIA is now available in SPSSx, SAS, and BMDP. All the 
three statistical packages provide UPGHA as an option of the cluster program and 
use an identical nmthod to coxnpute distances between clusters. The original 
dendrogram of the Clayton State Sample courses produced by SPSS-X is presented 
in Figure S-^IB. 

Courses were classified into 13 coursework patterns according to a 
hierarchical cluster structure. In fact, the choice to present the data in 13 
clusters is arbitrary. Any number of clusters can be identified depending on 
the hierarchical cluster structure produced; this structure remains constant 
regardless of the number of clusters used to form coursework patterns. A 
procedure for selecting the optimum number of clusters and for validating the 
resulting patterns will be described in greater detail in a subsequent section 
on the discriminant analysis of the coursework patterns in the Clayton State 
Sample. 

Using a 13 cluster solution to the quantitative cluster analysis, the 
largest number of courses are found in Coursework Cluster #2 with 33 courses* 
The smallest clusters are the 12th and 13th clusters with 2 courses. Overall, 
the differentiation between clusters is attributable to the number of criterion 
variables used in the analysis and also to the choice of those variables. The 
cluster analysis and subsequent discriminant emalysis suggested that student 
residuals on GRE item- types are strong , reliable and robust measures in 
differentiating student general learned abilities. 
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The hierarchical cluster structure is presented in the dendrogram sunjnary 
of Figure 6-8. For concise visual presentation, the complex sub-structures of 
each of the clusters were omitted from the dendrogram. The dendrogram displays 
the clusters being combined and the distances between the clusters at each 
successive step, suggesting that the 13-cluster solution examined is appropriate 
and interpretable . Cluster analyses using smaller and larger numbers of cluster 
groupings provided conparably high levels of correct classification, as deter- 
mined by subsequent discriminant analyses. However, as the resemblance index 
increases (Euclidean distance betweencourses), more distant courses are joined 
into larger and larger clusters. A 5-cluster solution, for example, provides a 
high degree of aggregation which may prove to have a high degree of predictive 
validity but a low level of utility in differentiating coursework by item-type. 
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Figure . SPSS-X Dendrogram. 
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RESCALED DISTANCE CLUSTER COMBINE 



CASE 
LABEL 



PSV 204 
PSV 303 
PSV 301 
ENG 201 
PSV 416 
BI0L1 1 1 
BIOLlia 
PSV 101 
PSV 423 
HIST 1 13 
ENG 313 
PSV 202 
PSV 203 
CIS 480 
MK 430 
HIST252 
PHE0170 
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Figure 6-7. SPSS-X Dendrogram. 
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Figure 6-7. SPSS-X rendrogram. 
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Figure 6-7. SPSS-X Dendrogram. 
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Figure 6-8. Dendrogram of hierarchical cluster structure for Clayton Stat© 
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f.gure fa-ya. Courses within coursework clusters (13-cluster solution): Clayton 
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••*" indicates a course misclassif ied according to the discriminant analysis 

of course clusters. 
"V' indicates it was a Clayton State course; all other courses are Georgia 

State courses. 
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Figure 6-9b. Courses within coursework clusters (13-cluster solution): Clayton 
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indicates a course misclassif ied according to the discriminant analysis 
of course clusters. 

indicates it was a Clayton State course; all other courses are Georgia 
S+'ate courses. 
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Figure 6-9c. Courses within coursework clusters (13-cluster solution): Clayton 
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*•*" indicates a course misclassif ied according to the discriminant analysis 
of course clusters. 

indicates it was a Clayton State course; all other courses are Georiga 
State courses. 



Observations about the Clusters 

A careful exandnation of courses within each cluster see:3is to indicate that 
some courses coming from the same department appear in the same cluster, such as 
the Psychology courses (Psy) in Cluster #8. Similarly, there are apparent 
sequences of courses, such as the Math 211, 212, 215, 216 sequence in Cluster 
#5. Also, a set of courses coming from various related disciplines may form a 
homogeneous cluster on the basis of a set of given attributes or criteria. The 
homogenity of disciplines is less apparent in Cluster #9. Nevertheless, the 
humanities and social. sciences seem to predominate this group. 

At this point in the analysis, it is difficult to describe which dimensions 
of student general learned ability each cluster represents. However, it seems 
to be clear that one pattern of course enrollment may contribute to student 
general learned ability in a way significantly different from the other course- 
work patterns. Supporting this i- a more detailed examination of subset courses 
of each clusters. In many cases, those courses offered at the same level tend 
to be combined into pairs together. But, those pairs are agglomerated with 
other courses offered at the higher level again according to the hierarchical 
structure of clusters. This may suggest that student gains in general learned 
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abilities may be obtained through a sequential enrollment pattern during the 
college years, not at a single stage of the sequence (such as the freshman year 
experience) . 
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Discriminant analysis of coursawork patterns 

In examining the dendrogram of the Clayton State Sample, a logical que^stion 
arises as to which nuziiber of clusters or pattern groupings provides the best 
explanation of the relationship between student item-^type score gains and 
coursework patterns. Separate discriminant analyses of different numbers of 
cluster groupings were be perfonued in order to determine the nundoer of 
groupings that optimdzes the proportion of courses correctly classified* Four 
different cluster solutions provided ccHcparably high levels of correct 
classif ication : 

8 cluster solution : 92,66% of courses correctly classified 

9 cluster solution : 92.09% of courses correctly classified 
n cluster solution : 90.96% of courses correctly classified 
13 cluster solution : 89.83% of courses correctly classified 

While these cluster solutions produced coitparable classification results, 
the different grouping evidenced differing effectiveness in laentifying 
relationships between mean item- type residuals and coursework patterns. The 
13-cluster solution was used in this report. 

The discriminant analysis was conducted using the DISCRIMINANT program in 
SPSSx in the following manner. Using the course item^type attributes as inde- 
pendent variables and the cluster group membership as the dependent variables^ 
discriminant functions were applied to the data. The discriminant functions and 
the courses item-type mean residual scores were used to see how correctly the 
discriminant function identifies each cluster group. The resulting percentage 
of correct predictions serves as a secondary validation of the cluster solution 
(Bradfield and Orloci, 1975; Green and Vascotto, 1978; Romesburg^ 1984). 
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Figure 6-10* DiKriAliumt §xadjBiM of Iht IS-^urttt: •olutioct for ClartoD Stau S«fdB 
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6.1% 93.9% .0% .0% .0% .0% .0% .0% .0% .0% .0% .0% .0% 

Group 3 18 00 17 00000001 00 
.0% .0% 94.4% .0% .0% .0% .0% .0% .0% .0% 5.6% .0% .0% 

Qsoap 440103000000000 
.0% 25.0% .0% 75.0% .0% .0% .0% .0% .0% .0% .0% .0% .0\ 
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.0% .0% .0% .0% .0% .0% .0% .0% .0% .0% .0% 100.0% .0% 

Group 13 20010000000001 
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Percent of "Grossed" Clusters correctly classified: 89.83% 
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Correlations of item- types and aiscriminant functions 

The discriminant analysis of Clayton State Sanple provided secondary vali- 
dation that 89.83% of the classification of courses was correctly predicted by 
the cluster analysis (See Figure 6-10). The discriminant analysis is a 
secondary validation, since it is based on the same saniple of transcripts and 
test scores. Stated sinqply, almost 9 of 10 courses inost frequently taken by 
Clayton State Sample students were correctly classified according to their mean 
residual GRE scores. 'While the cluster analysis produces coursework patterns 
according to criteria of general student learning, additional steps are needed 
(1) to determine which courses were correctly classified and (2) to ascertain 
which item-type scores contributed to any given coursework pattern. 

Using the BREAKDOWN procedure in the DISCRIMINANT program of SPSS-X 
(Norusis, 1985), courses whi h were incorrectly classified or which may be 
classified within another coursework pattern are identified. These courses are 
marked with an in Figure 6-14. Vftien a set of data is taxonomized by 
multiple and independent criteria, it is reasonable to expect that the first 
groupings are generally the most homogeneous and the later groupings are the 
most heterogeneous. Consequently, the highest proportions of misclassif ications 
occurred in Cluster #5, while Clusters #3, #7, #10, and #12 had relatively few 
misclassif ications. 

To compute the contribution of each mean item-type residual score to the 
discriminant functions, the correlation coefficients between mean residual 
scores and discriminant functions were examined. Figure 6-11 shows the pooled 
within-group correlations for the 13-cluster solution of Clayton State Sample 
coursework. 
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flgura 6-11. PooXid wlthla-gra* oomOatioo bttWMm mbd it«-tjpe resiaaal toomM «od ai»crt»lnaDt 
fuDctioQi for UaytoD SUtt Safde. 
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Correlations of coursework clusters and discriminant functions 

Figure 6-11 summarizes relationships between GKE item-type residuals and 

discriminant functions: 

Function 1 was not strongly correlated with the item-types; 

Function 2 was positively correlated to Quantitative Comparisons (r=.82), 
and was positively correlated to Analytic Reasoning (r=.52); 

Function 3 was positively correlated to Logical Reasoning (r=.53), and 
was positively correlated to Antonyms (r=.51); 

Function 4 was positively correlated to Antonyms (r=.62), and 

was positively correlated to Regular Mathematics (r=.67); 

Function 5 was positively correlated to Sentence Coscpletion (r=.57), and 
was positively correlated to Analogies (r=.58); 

Function 6 was not strongly correlated with the item- types; 

Function 7 was positively correlated to Analogies (r=.50); 
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Function 8 was positively correlated to Logical Reasoning Cr«.53), and 
was positively correlated to Data Interpretation**(r=^.61) ; 

Function 9 was not strongly correlated with the itein-types. 
The pooled within^^group correlations established relationships between the 
discriminant functions and the GRE item-type residuads. Each discriminant 
function explains a certain proportion of the variation in residual scores. 

Discriminant functions with strong explanatory power, ''good discriminant 
functions, hi.^ve large between-^cluster variability and low within-cluster 
variability (Haggerty, 1975; Norusis, 1985). The eigenvalues of Figure 6-12 
present the ratio of between-group to within-group sums of squares of the 
residuals. Large eigenvalues are associated with the discriminant functions 
that most contribute to explaining variability in GRE item-type scores. 

Vfilk^s Lanibda is the ratio of the within-group sum of squares to the total 
sum of the squares. It represents the proportion of the total variance in the 
discriminant function values not explained by differences among cluster groups. 
Viilk's Lambda serves as a test of the null hypothesis that there is no 
difference in the mean residuals of a coursework cluster means and the mean 
residual scores of the coursework in the total san?>le. Thus, the eigenvalues 
and canonical correlations indicate the extent to which each discriminant 
function contributes to our understanding of the variability in coursework mean 
residuals. Lanibda test the null of the differential coursework hypothesis for 
each discriminant function. Functions 1 to 4 explainer 89 percent of score 
variance. Function 1 explained 39 p»ercent of variance; Function 2 explained 25 
percent of variance. Significant differences occurred by cluster means and 
sample means affirming the differential coursework hypothesis. 
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Figure 6-12. Canonical discriminant functions: Clayton State San?>le. 
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Figure 6-13. Canoclcal dUariminant functioM •v*lu«t«a «t ore»?> mu 
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Interpreting the coursework clusters for the 13-cluster solution 

Figure 6-13 shows the coursework cluster means (group centroids) for each 
discriminant function. Clusters with positive or negative means greater than 
1.0 were selected for further analysis. Clayton transfer students who took 
different Clayton and Georgia State coursework showed different types and 
degrees of iit^srovement in general learned abilities, 

Courseiiork Cluster §1 had high positive means on Functions 2 and 4^ and a 
high negative mean on Function 3. Function 2 was positively correlated with 
Quantitative Con5>arisons (r^,82) and Analytic Reasoning (r=^,52). Function 3 was 
positively correlated with Logical Reasoning (r=.53) and Antonyms (r=.51). 
Function 4 was positively correlated to Antonyms {r-.62) and Regular Mathematics 
(r-.67). Therefore^ students who enrolled in the coursework pattern represented 
in Cluster #1 iir5>roved in ability on Quantitative COT^sarisons^ Analytic 
Reasoning, and Regular Mathematics but declined on Logical Reasoning 
item- types. The results for the item-type of Antonyms were inconclusive. This 
cluster contained Accounting, Business, Decision Mathematics, Management, and 
Real Estate; only 2 of the 16 courses were tgiken at Clayton State. 

Cluster #2 had a high positive mean on Function 2. Students enrolling in 
this set of courses showed high gains in Quantitative Comparisons and Analytic 
Reasoning. Accounting, Business Administration, Decision Mathematics, 
Economics, and Management coursework predominated the cluster; 12 of 33 courses 
were taken at Clayton State. 

Cluster 13 had a high positive mean on Function 2 and a high negative mean 
on Function 1. Function 1 was not strongly correlated with the item- types. 
Therefore, students enrolling in this cluster tended to show positive gains in 
their ability to answer Quantitative Coo^atrisons and Analytic Reasoning 
questions. Accounting 201, 202 and Econouiics 201, 202 were discernable 
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sequences of courses viithin this cluster. Unlike the preceding clusters, 
however^ there were not cleax links to a particular discipline or field of 
study; 14 of 18 courses in this cluster came from Clayton State. 

Cluster #4 had no high positive or negative means on Functions 1 through 4. 

Cluster #5 evidenced a high positive group mean on Function 3 and a high 
negative group mean on Function 2. This evidence suggested that students 
enrolling in Cluster #5 courses declined in ability on Quantitative CoB^^arisons 
and Analytic Reasoning but gained on Logical Reasoning and Antonyms item- types. 
Here the relationship between the score residuals and coursework were 
heterogeneous* Music and Mathematics coursework predominated the group. Only 
Philosophy 211 was a Clayton course. 

Cluster #6 had 3 hi^ih positive means on Functions 3 and 4, and a high 
negative mean on Function 1. Students enrolling in this coursework gained in 
Logical Reasoning, Antonyms, and Regular Mathematics. Four of the five courses 
were taken at Clayton State. 

Cluster #7 had a high negative group mean on Function 2. Students 
enrolling in Cluster #7 declined on Quantitative Comparisons and Analytic 
Reasoning item- types. Journalism, Psychology, and Spanish coursework 
predominated the cluster; 6 of 15 courses were taken at Clayton State. 

Cluster #8, Cluster #9, and Cluster flO had no high positive or negative 
means on Functions 1 through 4. 

Cluster ill had a high negative group mean on Function 4. Students 
enrolling in Cluster #11 declined in abilities relative to Regular Mathematics 
and Antonyms. English and Psychology coursework predominated the cluster; all 
courses were taken at Clayton State. 

Cluster tl2 consisted of two courses. Therefore^ no further analysis was 
conducted. 
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cluster tl3 had no high positive or negative means on Functions 1 through 

4. 

Figures 6- 14a — 6-14f portray the coursework clusters and the mean residual 
item- type with which they were found to be associated. It should be cautioned 
that the association was established at the cluster level • No direct causal 
link is intimated between student enrollment in any one given coiurse and scores 
on the GRE. Furthermore, at this point, one cannot say why students who 
enrolled in these courses had higher residual scores. The cluster serves to 
hypothesize relationships between coursework patterns and the general learned 
abilities measures by the item-types of the GRE, One can say that students who 
enrolled in specific patterns of coursework tended to iiqprove on specific 
item-^types within the GRE^ while others who enrolled in different coursei#ork 
patterns did not tend to show such improvement. This evidence affirms the 
hypothesis that student gains in general learned abilities are associated, 
positively and negatively, with the coursework in which they enrolled. Further 
analysis is required to determine the nature of these associations. 
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Figure 6-14a, Cluster 1: High positive mean residuals on Quantitative Cocpari-- 
sons (QC), Analytic Reasoning (ARE), and Regular Mathematics (RM); 
high negative residuals on Logical Reasoning (LR). 

)ESCRIPTION MISCIASSIFICATIONS 

Principles of Accounting I 
Survey of Art I 

Introduction to International Business 
Decision Mathematics III * 
Decision Mathematics III 
Dynamics of Labor Problems 
Title unavailable. 
European Literature II 
Introduction to Data Processing 
Intro to Personnel & Industrial Relations 
Organization Theory and Practice 
Operations Management 
Introduction to Philosophy 
Residential Real Estate Valuation 
Urban Development Regulations 

(Title unavailable.) * 



DEPT 


HUM 




:sss=S! 


AC 


201 


ART 


178$ 


BA 


309 


DM 


231 


DM 


310 


EC 


386 


ENG 


20 


ENG 


202 


IS 


201 


MGT 


430 


MGT 


435 


MGT 


470 


PHIL 


201 


RE 


410 


RE 


495 


SCPH 


10$ 



indicates a course misclassif led according to the discriminant analysis 
of course clusters. 

indicates a Clayton State course; all other courses are Georgia State 
courses . 
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Figure 6- 14b. Cluster 2: High positive mean residuals on Analytic Keasonin 
(ARE) and Quantitative Co&;>ari5ons (QC)* 

s s s: s s; s r£ a s ss » ss 35 ss as SIS SIS & s = 9C s ^ s s «a« s;:: ssss sss a ss B ss 

CLUSTER DEFT WJM DESCRIPTION MISCLASSIPICATION 
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Principles of Accounting I 
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Principles of Accounting II 
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AC 
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(Title unavailable.) 
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Basic Accounting Systems 
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AC 
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Financial Accounting I 
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Financial Accounting II 
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BA 
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Intro to Managetnent Information Systems 
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Strategic Hanagcmient/Policy 
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Office Machines/Business Machines I 
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CNST 
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(Title unavailable • ) 
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Decision Mathematics II 
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DM 
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Decision Mathematics IV 
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Survey of Calculus 
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Introduction to Business Statistics 
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General Modeling Techniques w/ Applications 
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( Title unavailable . ) 
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Prin. of Macro-Economics/Prin. of Econc^nics 
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Prm, of Micro-Econ«nics/Economic Problems 
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Money and Credit 
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Corporate Finance 
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Intro to Risk Management & Insurance 
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(Title unavailable.) 
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Legal Environment of Business 
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Management Concepts Theory & Practice 
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Intro to Organizational Behavior 
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301 


Basic Marketing 
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PED 
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(Title unavailable.) 
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PHIL 
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Introduction to Logic 
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PROG 


20$ 


(Title unavailable.) 




RE 


301 


Real Estate Principles 


2 


SPCH 
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Public Speaking 



indicates a course misclassif led according to the discriminant analysis of 
of course clusters. 
"$" indicates a Clayton State course; all other courses are Georgia State 
courses. 
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Figure 6-14c. Cluster 3: High positive mean residuals on Quantitative 
COTiparisons (QC) emd Analytic Reasoning (ARE). 

CLUSTER DEPT NUM DESCRIPTION MISCIASSIFICATIONS 
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ACCT 


201$ 


Principles of Accounting I 
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ACCT 


202$ 


Principles of Accounting II 
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BIO 


142 


Organisms and their Environment 
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CHEM 


112$ 


Chemical Principles II 
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COMP 


201$ 


(Title unavailable.) 
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Prin. of Macro-Economics/ Pr in. of Economics 


3 


ECON 


202$ 


Prin. of Micro-Economics/Economic Problems 
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ENG 


111 


CoB^>osition I 


3 


LSM 


436 


Selection & Utilization of Education Media 


3 


MATH 


111$ 


(Title unavailable.) 


3 


MATH 


121$ 


(Title unavailable.) 
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MUSI 


211$ 


(Title unavailable.) 


3 


PHED 


159$ 


(Title unavailable.) 
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POLI 


111$ 


(Title unavailable.) 
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PSY 


20$ 


(Title unavailable.) 
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SOCI 


105$ 


(Title unavailable.) 


3 


SPOT 


121$ 


(Title unavailable.) 


3 


SPCH 


150 


(Title unavailable.) 



indicates a course misclassif ied according to the discrimiing to the dis- 
course clusters. 

indicates a Clayton State course; all other courses are Georgia State 
courses. 



ERIC 



- 139 



15 



Figure 6-14d. Cluster 5: High positive mean residuals on Xiogical Reasoning 
(LR) and Antonyms (ANT)» 

CLUSTER DEPT NUH DESCRIPTION 
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5 


APVC 


200 


Applied Music-Lower Level-Freshman/ Sophocnore 


5 


APVC 


300 


Applied Music-Concentration 
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CIS 


303 


Assembly Language Progranming 


5 


FED 


305 


Human Growth & Development 
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Educational Psychology 
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Principles of Conputer Progranming 
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MATH 


211 


Calculus of One Variable 
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HATH 


212 


Calculus of One Variable 
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MATH 


215 


Multi-Variable Calculus 
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MATH 


216 


Intermediate Calculus 


5 


MUS 


102 


Voice Laboratory 
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MUS 


108 


Choral Ensemble 


5 


MUS 


110 


Concert Attendance 


5 


PHIL 


211$ 


(Title unavailable.) 



indicates a course misclassif ied according to the discriminant analysis of 
course clusters. 

indicates a Clayton State course; all other courses are Georgia State 
courses . 
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Figure 6-14e. Cluster 7: High negative mean residuals on Quantitative 
Comparisons (QC) and Analytic Reasoning (ARE). 

CLUSTER DEPT NUM DESCRIPTION MI SCIASSIFI CATIONS 
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(Title unavailable.) 
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BIO 


101$ 


(Title unavailable.) 
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FILM 
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History of the Motion Picture 
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101 


Physical Geology 
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JOUR 
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300 Years of American Journalism 
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News Writing & Reporting 
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Copy Reading 
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301 
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(Title unavailable.) 
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Child Development 
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(Title unavailable.) 
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SPAN 


101$ 


£l«nentary Spanish 
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(Title unavailable.) 



indicates a course misclassif ied according to the discriminant analysis of 
course clusters. 

"$" indicates a Clayton State course; all other courses are Georgia State 
courses. 



Figure 6-14f. Cluster 11: High negative mean residuals on Regular Mathematics 
(RM) and Antonyms (ANT). 
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MISCLASSIFICATIONS 












11 


EDUC 


201$ 


Title unavailable ♦ 




11 


ENGL 


111$ 


CcOTposition I 




11 


ENGL 


112$ 


Composition II 




11 


PSYC 


201$ 


General Psychology 




11 


PSYC 


258$ 


Title unavailable* 





indicates a course misclassif ied according to the discriminant analysis of 
course clusters. 

"$'• indicates a Clayton State course; all other courses are Georgia State 
courses . 
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VII. CoCTTOon Course Numbering at Clayton State 
and Georgia State 



Are the credits a student earns at a conanunity college truly equivalent to 
those earned in the lower division of a university? This : Jnple question drives 
efforts to ease, increase emd facilitate coniminity college student progress, 
persistence, performance and degree attainment in baccalaureate programs 
(Giddings, 1985; Richardson & Doucette, 1980). Recent examination of the 
quality of undergraduate education, of the distribution of funding among public 
higher education, and of the success of minority students has brought a renewed 
interest in the process of articulation of credits between ccatsminity and junior 
colleges to four-year college and university baccalaureate programs (Eaton, 
1990). 

Two coinmon means for facilitating the transfer process within public higher 
education systems have been the common core curricula and common course number 
schemes (Kintzer & Wattenbarger, 1985). A core curriculum asks students at all 
system colleges and universities to take the same sequence of courses to 
coniplete their general education requirement for the baccalaureate degree. When 
a student con?5letes the core at one institution, it is held applicable to degree 
requirements for all system institutions (Morgan s; Teel, 1990'. The underlying 
assumption is that the effect of the core pattern of courses at one institution 
in the system is con^arable to all others in its effect on student learning. 
Similarly, a common course numbering system requires that comparably named 
courses bear the same department and course numbering scheme. The assumption is 
that Math 101 at Clayton State is comparable in its effect on student learning 
to Math 101 at the Georgia State University. 
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What is the most effective pattern of undergraduate general education for a 
given group of students? "No curricular concept is as central to the endeavors 
of the American college as general education, and none is so exasperating ly 
beyond the reach of consensus and understanding" (Carnegie Foundation for the 
Advancement of Teaching 1977, p. 164). The debate has continued concerning the 
structure and content of general education as discussed by numerous reports 
(Association of American Colleges 1988; National Institute of Education 1984; 
National Endowment for the Humanities, 1984; American Association of Colleges 
Consnittee's Project on Redefining the Meaning and Purpose of Baccalaureate 
Degrees 1985). Yet, evidence has emerged that different students experience 
different subenvironments within colleges and universities, particularly in 
relation to their formal coursework (Pascarella, 1985; Ratcliff, 1989; Jones & 
Ratcliff, 1990). 

At one end of the continuum, there are advocates for a core curricv.ium who 
believe that general education should consist of prescribed coursework riKjuired 
of all students (Boyer & Kaplan 1977; National Endowment for the Humanities 
1989). They believe that one curriculum is appropriate and fits all students. 
Others support the distributive model which consists of "requirements designed 
to ensure that each student takes a minimum number of courses or credits in 
specified academic areas" (Levine, 1978, 11). Students at many colleges meet 
distribution requirements by enrolling in courses selected from many offerings 
in differents subject fields. The advocates of the distributive requirements 
believe that different curricula are necessary for different students based upon 
student interest and/or student ability. Common course numbering and system 
wide core curriculum requirements are based on the assun^tion that the effects 
of 'omroonly named and labeled courses are the same. This section examines 
stv.dent transcripts and test scores of Georgia State (native) and Clayton State 
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(transfer) students to determine the extent to which general education 
coursework with cOT?)arable course numbering produces consnon effects in the 
general learning abilities among these college students. 

Problem Investigation 

Given the views of the advocates of comon course nunibering systems as 
articulation mechanisms^ the fundamental question is whether the effect of 
coursework at a two-year college (Clayton State) is con5>arable in its effect on 
general learned abilities to that of the identically numbered coursework at an 
urban doctoral granting university (Georgia State) within the same state system 
of higher education. We first established relationships between student 
coursework and comnion measures of general learned abilities^ the Scholastic 
Aptitude Test and the General Test of the Graduate Record Examination. 
Secondly, we examined if these relationships were the same for Georgia State 
native students (who began their education at this institution) and for Clayton 
State students • 

Sample 

Two stratified samples of graduatinq seniors were drawn frc^ Georgia State 
University (GSU). Since the sample size was small, they were combined 
together. Two subsamples were drawn frc«n this combined san^le. One sxibsample 
consisted of 76 students who had earned up to 90 quarter credits at a nearby 
public two-year college and subsequently transferred to GSU. The second 
subsample consisted of 168 "Georgia State^ students who earned their credits 
exclusively from this institution. These students graduated from GSU during the 
1986-87 and 1987-88 academic years. Analysis indicated that the sample was 
proportional to the distribution of Scholastic Aptitude Test (SAT) scores. 
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majors, and other socio- econanic characteristics of the population of graduating 
seniors at this institution. 

Differences in Clayton State and Georgia State Subsamples Characteristics 

A brief description of the characteristics of the subsasnples reveals scsne 
differences between the Clayton State and Georgia State groups. Gender is a 
factor related to academic performance. Approximately two- thirds (65.8%) of the 
Clayton State group were female, while 56.5 percent of the Georgia State group 
were female. 

Ethnicity is also related to academic performance. Ninety-two percent of 
the Clayton State group were white, while 35.1 i>ercent of the Georgia State 
students were white (see Figure 7-1). However, 47.6 percent of the Georgia 
State students did not indicate their ethicity. Gender and ethnicity 
differences may be contributors to the variation in performance among the tvo 
groups of students, but due to missing data, were not directly addressed in this 
analysis. 

Major field of study has been shown to be correlated to performance in the 
GKE examinations. The distribution of majors in the Clayton State group 
approximated that of the Georgia State group. Majors in Accounting, Journalism, 
Management, Marketing, and Psychology were frequently evident in both groups. 
These majors were dominant curricular groups at GSU and may have an effect on 
the variation in scores of general learning but did not vary significantly 
between the Clayton State and Georgia State students. 

Both subsaxcples enrolled in GSU coursework dispersed over a number of 
years. In the Clayton State group, one student began his/her enrollment in 1958 
while for the Georgia State group two students began their enrollment in 1970. 
Nearly one- third (30.4 percent) of the Georgia State students and one-quarter 
(24.8 percent) of the tranfer students began their enrollment prior to 1980. 
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Thus, there were jnore Georgia State students than Clayton State students 
enrolling in courses on a part-tinje basis 

Students in the Clayton State zmd Georgia State groups were clearly 
planning some form of post-baccalaureate study (see Figure 7-2). Over one-half 
(56.6%) of the Clayton State students and 66.1 percent of the Georgia State 
students intended to pursue a master's degree. Approximately 16 percent oi the 
Clayton State and Georgia State students wanted to enter a doctoral program. 
These students planned advanced study in greater proportion than nrast 
undergraduates and reflect the self -selected nature of the san?)le. 

The educational attainment of parents has been shown to be positively 
correlated to student achievement in college. One-quarter of the fathers and 
15.8 percent of the mothers of the Clayton State group had attained a 
high schools diploma or its equivalent while over 14.3 percent of the fathers 
and 30.9 percent of the mothers of Georgia State students had attained a high 
school diploma. Only 1.3 percent of the fathers and the mothers of Clayton 
State students had attained at least a bachelor's degree while 10.1 percent of 
the fathers and 9.5 percent of the mothers of Georgia State students completed 
the bachelor's degree (see Figure 7-3). 

Nearly two-thirds (63.2%) of the Clayton State students and 52.4 percent of 
the Georgia State students had performed some coosminity service during the past 
year, but for 42.1 percent of the Clayton State students and 38.1 percent of the 
Georgia State students this con5>rised less than five hours per week (see Figure 
7-4). Over one-third (38%) of the Clayton State students and 50 percent of the 
Georgia State students had earned some form of professional, community service, 
literary, artistic, or student government honor, or award. 

Contrary to popular conceptions of consnunity college and university 
students, the Georgia State students were more likely to be from a racial/ethnic 
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minority and were slightly more likely to be part-tijne students. For this 
reason, extrapolation of these results to other coitmunity college or university 
populations should be viewed with caution. 
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Figure 7-1* Distribution of Subsamples by Ethnicity 

CLAYTON STATE GEORGIA STATE 

ETHNICITY N Percent N Percent 



Not specified 1 1.32% 80 47.62% 

Black 2 2.63% 3 1.79% 

Chinese American 0 .00% 0 .00% 

Japanese American 0 .00% 0 .00% 

Other Asian American 1 1.32% 0 .00% 

Native American 0 .00% 0 .00% 

Chicano/Hispanic 2 2.63% 1 .60% 

White 70 92.11% 59 35.12% 

Foreign 0 .00% 0 .00% 

TOTALS 76 100.00% 168 100.00% 



Figure 7-2. Degree Objectives of Subsairj>les 

CLAYTON STATE GEORGIA STATE 

DEGREE OBJECTIVES N Percent N Percent 



Unknown 


9 


11.84% 


21 


12.50% 


Nondegree study 


8 


10.53% 


5 


2.98% 


Masters degree 


43 


56.58% 


111 


66.07% 


Intermediate degree 


3 


3 . 95% 


3 


1.79% 


(e.g. Specialist) 










Doctorate (Ph.D., Ed.D. 


) 12 


15.79% 


28 


16.67% 


Postdoctoral study 


1 


1.32% 


0 


.00% 


TOTALS 


76 


100.00% 


168 


100.00% 
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Figure Educational Attainment of Parents of Subsairples 

CIATTON STATE 



GEORGIA STATE 



HIGHEST LEVEL OF 
EDUCATION COMPLETED 


N 


Father 
Percent 


N 


Mother 
Percent 


Father 
N Percent 


Mother 
N Percent 


No response 


6 


7 


.89% 


6 


7.89% 


12 


7. 


14% 


13 


7.74% 


Grade school or less 


10 


13 


.16% 


14 


18.42% 


22 


13. 


10% 


12 


7.14% 


Some high school 


15 


19 


.74% 


29 


38.16% 


21 


12. 


50% 


30 


17.86% 


High school diploma or equivalent 


19 


25 


.00% 


12 


15.79% 


24 


14. 


29% 


52 


30.95% 


Business or trade school 


9 


11 


.84% 


3 


3.95% 


25 


14. 


88% 


15 


8.93% 


Some college 


7 


9 


21% 


5 


6.58% 


18 


10. 


71% 


12 


7.14% 


Associate degree 


6 


7 


.89% 


3 


3.95% 


10 


5. 


95% 


7 


4.17% 


Bachelor's degree 


1 


1 


.32% 


1 


1.32% 


17 


10. 


12% 


16 


9.52% 


Some graduate or professional school 


1 


1 


.32% 


2 


2.63% 


6 


3. 


57% 


1 


.60% 


Graduate or professional degree 


2 


2 


.63% 


1 


1.32% 


13 


7. 


74% 


10 


5.95% 


TOTALS 


76 


100 


.00% 
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Fig\ire 7-4. Community Service Activities of Subsamples 

HOURS PER WEEK 
IN COMMUNITY SERVICE 
ACTIVITIES OVER THE PAST YEAR 



CLAYTON STATE 
N Percent 



GEORGIA STATE 
N Percent 



No response 
0 hours 

I- 5 hours 
6-10 hours 

II- 20 hours 
More than 20 
TOTALS 



6 7.89% 

28 36.84% 

32 42.11% 

6 7.89% 

1 1.32% 

3 3.95% 

76 100.00% 



19 11.31% 

61 36.31% 

64 38.10% 

16 9.52% 

2 1.19% 

6 3.57% 

168 100.00% 
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Reliability and Correlation of Ite > n"types 

On average, the Clayton State group answered 93 of 186 items correctly (see 
Figure 7-5); the Georgia State group gave correct responses to an average of 100 
of the 186 items (see Figure 7-6). Based on raw GSE scores alone, the Georgia 
State students performed better than the Clayton State students. This 
performance may be attributable to differences in incoming student ability or to 
the extent of gain in learning over the four years of undergraduate education. 



Figure 7-5. Distribution 


of GRE 


Scores for 


students. 


Clayton State Group of GSU 
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Figure 7-6- Distribution 


of GRE 


Scores for Native Students, 


Georgia 


State Group 
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Differences among scores for the subsamples appeared when the effect cf tie 



precollege learning (as measured by the SAT) was removed. VJhen the theoretical 
scores (as predicted by corresponding SAT scores) were compared with the 
students' actual responses, the subgroups showed large proportions of change on 
most item- types. Figure 7-7 presents the results of the regression analyses of 
individual GRE item- type scores on SAT siibscores. For both the Clayton State 
and Georgia State groups, the greatest amount of variance in item-type 
residuals, including the greatest standard error and standard deviation, was 
found in Analytic Reasoning. In analysis of other student groups and 
institutions, the greatest amount of score variance was in Analytic Reasoning as 
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well (Ratcliff, 1987, 1988; Jones & Ratcliff, 1990). The variance in the 
residuals holds implications for the ensuing cluster analysis in that GRE 
item-types with greater variance will play a more significant role in sorting 
courses into clusters. As was discovered in th„ previous analysis of another 
institution, those GRE item-ti^es with smaller variance play less of a role in 
discriminating course clusters. 

Figure 7-7 compares the explained variance (r'"") for each GRE item-type, raw 
GRE sub-score and converted GRE sub-score. In all cases within the subsamples 
errors estimates were less than .01 and the SAT accounted for more variance in 
GRE sub-scores than in the GRE item-type scores. As this figure demonstrates, 
from 12.6 percent (Data Interpretation) to 38.45 percent (Reading Comprehension) 
of GRE item- type score variation among the Clayton State group was explained by 
SAT scores; from 26.9 percent (Data Interpretation) to 52.1 percent 
(Quantitative Comparisons) of GRE item- type score variation among the Georgia 
State group was explained by SAT scores. 

Using the student residuals obtained from the regression analysis above, 
the mean residuals for each course enrolling 5 or more students were calculated 
for all the 9 GRE item-types. Such a procedure did not assume that the specific 
gains of the students enrolled in each course were directly caused by that 
course. Rather, the residuals of each student were attributed to all the 
courses in which they enrolled, and the mean residuals for each course served as 
a proxy measure of student improvement in general learning. Once courses were 
clustered by these residuals, then hypotheses were generated and tested as to 
why students who enrolled in a given pattern of courses experienced significant 
improvement on one or more of the outcomes criteria (i.e., the item-type 
iesiduals) . 
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Figure 7-7. Summary of Regression Analysis of GRE Item-types on SAT Subscores 
for the Clayton State and Georgia State Groups 

Dependent 

Variables: Clayton State Group Georgia State Group 

76 Students 1&8 Students 

GRE Item-types on Adjusted Adjusted 

SAT Sub-scores CODE F Value Prob>F R-Squared F Value Prob>F R-Squared 
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Reading 
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DI 


11 
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62 
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.2686 


Analytic Reasoning 
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28 


.346 


.0001 


.2672 


99 


,616 


.0001 


.3713 


Logical Reasoning 


LR 


18 


.551 


.0001 


.1896 


74 


,640 


.0001 


.3060 


Raw Sub- test Scores 




















Verbal 


GRE-V 


62 


.267 


.0001 


.4496 


266 


.909 


.0001 


,6142 


Quantitative 


GRE-Q 


51 


.195 


.0001 


.4009 


268 


.383 


,0001 


.6155 


Analytical 


GRE-A 


37 


,490 


.0001 


.3273 


143 


.057 


.0001 


.4596 



Transcript Analysis of Clayton State and Georgia State Subgroups 

This section reports the use of the quantitative cluster analytic procedure 
to analyze the Clayton State and Georgia State groups. The results for each 
subsample are compared to determine the extent to which students benefit from 
different coursework patterns. Secondary validation ( discrijnninant analyses) of 




the two subsamples suggested that the cluster analytic mo&el was valid 
(secondary validity) and reliable means for determining coursework associated 
with the general learned abilities of undergraduates. The objects of these 
analyses are the courses which constitute the enrollment patterns of students in 
the subsamples. 

There were 3,427 courses listed on the 76 transcripts of the students in 
the Clayton State group, indicating that, on average, each of these students had 
enrolled in an average of 45.1 courses as part of the baccalaureate degree 
program. There were 1,088 unduplicated courses on the Clayton State 
transcripts, 177 in which 5 or more students had enrolled. These 17'' courses 
were the subject of subsequent quantitative cluster analysis. 

There were 7,850 courses listed on the 168 transcripts of thj students in 
the Georgia State group, indicating that, on average, each of these students had 
enrolled in an average of 46.7 courses as part of the baccalaureate degree 
program. There weie 1,244 unduplicated courses on the Georgia State 
transcripts, 300 in which 5 or more students had enrolled. These 300 courses 
were the objects of further analys..s. 



Discussion of Subgroup Residual Scores 

VJhlle the average of residuals means for the Clayton State group was 
positive, there were negative residuals on Antonyms and Quantitative 
Comparisons; positive residuals were particularly pronounced on Reading 
Comprehension (see Figure 7-8). The Georgia State group showed a positive 
average of mean residuals (see Figure 7-9). Negative residuals were found on 
the Antonyms, Regular Mathematics;, and Quantitative Comparisons item-types; 
positive residuals were particularly pronounced on the Reading Comprehension and 
Analytic Reasoning item-types. In both groups, there were positive and negative 
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residuals in comparable areas. Antonyms and Quantitative Con^jarisons were 
negative; Reading Comprehension was positive. Georgia State students also 
showed negative residuals on Regular Mathematics and positive residuals on 
Analytic Reasoning. 

vrhile the residual means describe the direction of change in general 
learned abilities (positive or negative), the standard deviation of residuals 
give estimates of the variation in change. The greatest variation in residuals 
occurred among the Georgia State subgroup. The greatest variation for both 
groups occurred in the Analytic Reasoning item-type. These data indicated 
differences in general learned abilities according to the entering SAT scores. 
Also, these data suggested that the effect of the undergraduate experience 
varied between the Clayton State subgroup and the Georgia State subgroup. 
Specifically, incoming ability as measured by the SAT accounted for less of the 
score variance among the Clayton State group. Using residuals as proxies for 
gains in general learned abilities, the Clayton State students showed greater 
change than did Georgia State students in all 9 areas measured by the GRE. 

The students in the two groups did not register strong positive 
improvement, once the effect of their precollege SAT scores were removed. 
Nevertheless, some students gained and some students declined in general learned 
ability within both subgroups. These cluster analyses differentiated between 
courses taken by students who showed gains on the item-types and those w)io 
declined. While the sum of ail residuals is zero, when residuals were 
aggregated by course, some courses had positive mean residuals while others had 
negative mean residuals for the students who enrolled in them. Courses with 5 
or more students had slightly positive average mean course residuals. This 
indicated that the average student did select common coursework associated 
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with gains in general learned abilities. 



Figure 7-8. Distribution of GRE Item-type Residuals Scores for 177 Clayton 
State Group Courses 

GRE Item-- types Nuinber Max Min Score Residual Standard 

of Items Value Value Range Means Deviation 
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Total 
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figure 7-9. Distribution of GRE Item-type Residuals Scores for 300 Georgia 
State Group Courses 

GRE Item-types Number Max Min Score Residual Standard 

of Items Value Value Range Means Deviation 
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3.51 
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10 
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-lO.Bti 
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9.56 


-1.70 


19.87 


.1521 


2.2364 


21 


4.02 


-4.59 


8.61 


.0658 


1.0998 


186 








.5261 





Creating the Raw Data Matrix and the Resemblance Matrix for the Clayton State 
and Georgia State Groups 

Uriing the mean residuals of the Clayton State group and the 177 courses 
found on 5 or more of their student transcripts, a raw data matrix was created. 
The data matrix consisted of 177 columns and 9 rows (177 x 9). Using the mean 
residuals of the Georgia State group and the 300 courses found on 5 or more of 
their student transcripts, a second separate raw data matrix was created. This 
data matrix consisted of 300 columns and 9 rows (300 x 9). The rows represented 
the criterion variables: the 9 GRE item-type residual scores. The columns 
represented those courses enrolling 5 or more students. Thus, each cell value 
of the matrix was a mean GRE item- type score gain for those sample group 
students enrolling in a specific course. 
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For the Clayton State group, a reseniblance matrix was created next to 
describe how closely each course resembled the other 176 courses according to 
the criterion variables: the student score residuals. Likewise, for the 
Georgia State group a resemblance was created to describe how closely each 
course resembled the other 185 courses according to the criterion variables. To 
ccilculate the resemblance matrix, the correlation coefficient was selected as a 
similarity measure. Thus, this coefficient assessed a pattern similarity of any 
two courses explained in terms of the 9 GRE item- type residuals. 

The resemblance matrix produced in this step consisted of 177 rows and 177 
columns for the Clayton State group and 300 columns and 300 rows for the Georgia 
State group, in which each cell value theoretically ranged from -1.00 to 1.00. 
The calculation of the resemblance matrix was done using the SPSSx PROXIMITY 
program. 

Selection of the Clustering Method for Clayton State and Georgia State Groups 

The method selected for the quantitative analyses w^s the average linkage 
method (UPGMA). The original dendrograms of both groups' courses were produced 
by SPSS-X. The results of the cluster analysis of the Clayton State group of 

is briefly described. Courses were classified into 13 coursework patterns 
according to the resultant hierarchical cluster structure. In fact, the choice 
to present the data in 13 clusters was arbitrary. Any number of clusters can be 
identified depending on t>ie hierarchical cluster structure produced; this 
structure remains constant regardless of the number of clusters used to form 
coursework patterns* A procedure for selecting the optimum number of clusters 
and for validating the resulting patterns is described in greater detail in a 
subsequent section of this report. 
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Using a 13-cluster solution to the quantitative cluster analysis, the 
largest number of courses were in Coursework Clusters #2 with 31 courses and 
Cluster #8 with 28 courses • The sraallest clusters were the 12th, and 13th 
clusters with 2 courses each^ Overall, the differentiation between cluster-. s was 
attributable to the nuinber of criterion variables used in the analysis and also 
to the choice of those variables. The cluster analyses and subsequent 
discriminant analyses for both groups suggested that student residual scores on 
GRE item-^types were strong, reliable aiid robust measures in differentiating 
student general learned abilities* 

Each hierarchical cluster structure was represented in a dendrogram. The 
dendrogram displayed the clusters being combined and the distances between the 
clusters at each successive step, suggesting that the 13-cluster solution 
examined is appropriate and interpretable. Cluster analyses using smaller and 
larger numbers of cluster groupings provided comparably high levels of correct 
classification, as determined by subsequent discriminant analyses. However, as 
the resemblance index increases (as the Euclidean distance between courses 
grows), n\ore distant courses joined into larger and larger clusters, h 
12-cluster solution, for example, might provide a high degree of aggregation 
which may result in a high degree of predictive validity but a low level of 
utility in differentiating coursework by item-type* 

For the Clayton State group, a careful examination of courses within each 
cluster indicated that some courses coming from the same department appear in 
the same cluster, such as the Psychology (PSY) in Cluster (see Figure 6-9). 
Similarly, there were apparent sequences of courses, such as the Math 211, 212, 
215, 216 sequence in Cluster #5. Also, a set of courses coming from various 
related disciplines may form a homogeneous cluster on the basis of a set of 
given attributes or criteria. 
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For the Georgia State group a 13-cluster solution was used for the 
quantitative cluster analysis* The largest number of courses were found in 
Coursework Clusters #1 with 53 courses and Cluster #6 with 50 courses. The 
smallest clusters were the 13th cluster with 3 courses and the 8th cluster with 
4 courses. 

For the Georgia State group, some courses from the same department appeared 
in the same cluster, such as the English (ENG) courses in Cluster #1, the 
Computer Information Systems (CIS) in Cluster #2, and the Journalism (JOURN) 
courses in Cluster #7 (see Figure I'-IO). Similarly, there were apparent 
sequences of courses, such as the Anthropology 201, 202, and 203 sequence in 
Cluster #1. Also, a set of courses coming from various related disciplines may 
form a homogeneous cluster on the basis of a set of given attributes or cri- 
teria. The homogeneity of disciplines is particularly apparent in Cluster #1* 
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Figure 7-lOa. Coursework Patterns: 13-Cluc' er for the Georgia State Group 
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indicates a course misclwssif ied according to the discriminant analysis of 
course clusters. 
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Figure 7- 10b. 


Coursework Patterns; 13-Cluster 


for the Georgia State Group 
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indicates a course misclassif ied according to the discriminant analysis of 
course clusters. 
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Figure T-lOc. 
Cluster 19 


Coursework Patterns: 13-Cluster for the Georgia State Group 
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Dir.criminant Analysis of Coursevs?ork Patterns for the Clayton State and Georgia 
State Groups 

If] examining the dendrograms of the Clayton State and the Georgia State 
groups, a logical question arises as to which number of clusters or pattern 
groupings provides the best explanation of the relationship between student 
item- type residuals and coursework patterns. Separate discriminant analyses of 
different numbers of cluster groupings were be performed in order to determine 
the number of groupings that optimizes the proportion of courses correctly 
classified for each group. Four different cluster solutions for the Clayton 
State group and for the Georgia State group provii^ed comparably high levels of 
correct classification. 
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While these cluster solutions produced comparable classification results, 
the different grouping evidenced differing effectiveness in identifying 
relationships between mean item-type residuals and coursework patterns. The 
13-cluster solution provides a great extent of information for the Clayton State 
and Georgia State groups about the relationships between these residuals and 
coursework patterns. It was therefore used in this research (see Figures 6-10 
and 7-11). 

As in the previous analyses, the discriminant analysis was conducted using 
the DISCRIMINANT program in SPSSx in the following manner. Discriminant 
functions were applied to the data using the course item-type attributes as 
independent variables and the cluster group membership as the dependent 
variables. The resulting percentage of correct predictions served as a 
secondary validation of the cluster solution (Romesburg, 1984). 
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Figure 7-11. Discritainaat analysis of the 13-cluster solution for t]» Georgia State Group 



Ictual lie. of T>redicted Qroup Heoiiejr^p 

Cluster Cases Gr 1 Gr 2 Gr 3 6r 4 8r 5 Gr 6 6r 7 Gr 8 Gr 9 Gr 10 Gr U Gr 12 Gr U 

Group 1 53 43. 3 0 0 0 0 0 0 0 0 0 1 0 
81.1% 5.7% .0% .0% .0% .0% .0% .0% .0% .0% .0% 1.9% .0% 

Group 2 49 0 45 1 1001 030000 
.0% 91.8% 2.0% 2.0% .0% .0% 2.0% .0% 6.1% .0% .0% .0% .0% 

Group 3 15 00 12 0 1 1 0 00 0200 
.0\ .0% 80.0% .0% 6.T% 6.7% .0% .0% .0% .0% 13.3% .0% .0% 

Group 49120600001 0000 
11.1% 22.2% .0% 66.7% .0% .0% .0% .0% 11.1% .0% .0% .0% .0% 

Gi cup 5 40 2 7 0 1 31 1 0 1 0 1 1 D :) 
5.0% 5.0% .0% 2.5% 77. S\ 2.5% .0% 2.5% .0% 2.5% 2.5% .0% .T% 

■oroup 6 50 2 7 2 0 0 38 0 0 0 0 1 00 
4.0% 14.0% 4.0% .0% .0% 76.0% .0% .0% .0% .0% 2.0% .0% .0% 

Group 7 14 01 0000 13 000000 
.0% 7.]% .0% .0% .0% .0% 92.9% .0% .0* .0% .0% .0% .0% 

Group 842000000200000 
50.0% ,0% .0% .0% .0% .0% .0% 50.0% .0% .0% .0% .0% .0% 

Group y 19 0300100 U 15 0000 
.0% 15.8% .0% ,0% 5.3% ,0^ .0% .0% 78.9% .0% .0% .0% .0% 

Group 10 12 03001 00 009000 
.U% 25.0% .0% .0% 8.3% .0% .0% .0% .0% 75.0% .0% .0% .0% 

Group 11 13 0 2 0 0 0 0 0 0 0 0 31 0 0 
.0% 15.4% .0% .0% .0% .0% .0\ .0% .0% .0% 84,6% .0% .01 

Group 12 19 1 200200001 0 13 0 
5.3% 10.5% .0% .0% 10,5% .0% .0% .0% .0% 5.3% .0% 68.4% .0% 

Group 13 3020000000 0 001 

.0% 66,7% .0% .0% .0% .0% .0% ,0% .0% .0% .0% .n% 33.3% 

Percent of "Grouped" Cluster correctly classified; 81.33% 
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Correlations and Discriminant Functions of Coursework Clusters 

The discriininant analyses of the GSU group provided secondary validation 
that 89.83 percent of the classification of courses was correctly predicted by 
cluster analysis for the Clayton State group while for the Georgia State group 
81.33 percent of the classification was correctly predicted* The discriminant 
analyses was a secondary validation^ since it was based on the same sainple of 
transcripts and test scores • 

Nine of ten courses most frequently taken by students in the Clayton State 
group were correctly classified according to their mean residual GRE scores 
while in the Georgia State group eight courses were correctly classified. While 
the cluster analysis produced coursework patterns according to criteria of 
general student learning, additional steps were needed (1) to determine which 
courses were correctly classified and (2) to ascertain which item-type scores 
contributed to any given coursework pattern. 

Using the BREAKDOWN procedure in the DISCRIMINANT program of SPSS-X 
(Norisus 1985) courses which were incorrectly classified or which may be 
classified within another coursework pattern are identified. To compute the 
contribution of each mean item-tvTe residual score to the discriminant 
functions, the correlation coefficients between mean residual scores and 
discriminant functions were examined. 

The relationships between GRE item-type residuals ano discriminant 

functions for the Clayton State group were described in the preceding section 

(see Figure 6-11). The relationr.hip between GRE item-^type residuals and 

discriminant functions ara listed below for the Georgia State group: 

Function 1 was negatively correlated with Antonyms (r^".62), and 

was positively correlated with Ar^alytic Reasoning (r=,61); 
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Function 2 was positively correlated with Reading Comprehension {r=.59); 

Function 3 was positively correlated with Reading Coirprehension (r=.64); 

Function 4 was positively correlated with Analytic Reasoning (r-.57), and 

was positively correlated with Quantitative Contparisons {r=.50); 

Function 5 was positively correlated with Quantitative Comparisons (r=.67); 

Function 6 was positively correlated with Regular Mathematics (r=.65); 

Function 7 was positively correlated with Logical Reasoning (r=.74), 
was positively correlated with Analogies (r=.61), and 
was positively correlated with Sentence Coirpletion (r=.68); 

Function 8 was positively correlated with Data Interpretation (r=.76); 

Function 9 was negatively correlated with Sentence Completion lr=-.57). 
Once the relationships between discriminant functions and mean item-type resi- 
duals have been established, then the relationships between the discriminant 
functions and the coursework clusters can also be determined. 

By examining the average score of each cluster group for each discriminant 
function, the extent to which each discriminant function contributes to that 
group was calculated. Functions which had no correlation with specific 
item-type residuals were omitted. 

Each discriminant function explains a certain proportion of the variation 
in residual scores. Discriminant functions with strong explanatory powsr, "good 
discriminant functions," have large between-cluster variability and low 
within-cluster variability (Romesburg, 1984). The eigenvalues of Figures b-12 
and 7-12 present the ratio of between-group to within-group sums of squares of 
the residuals. Large eigenvalues are ascociated with the discriminant functions 
that most contribute to explaining variability in GRE item- type scores. 

VJilk's Lambda is the ratio of the with-group sum of squares to the total 
sum of the squares. It represents the proportion of the total variance in the 
discriminant function values not explained by differences among cluster groups. 
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le&idual scores ox the coursework in the total sample. 

Thus, the eigenvalues and canonical correlations indicate the extent to 
which each discriminant function contributes to our understanding of the 
variability in coursev^ork mean residuals. Lambda tests the null of the 
differential coursework hypothesis for each discriminant function. Results of 
the analysis indicated a relationship did exist between coursework taken and 
performance on the GRE. Certain GRE item*- type residual scores predominated. 
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State and Georgia State Subgroups 

Coursework clusters for the Clayton State group were analyzed and 
terprcted in the preceding chapter. Georgia State native student coursework 
cr^.-ters with positive or negative means greater than 1.0 were selected for 
further analysis. Coursework Cluster #1 had a high negative group mean on 
Function 1 and a hiyh positive group mean on Function 2. Function 1 was 
positively correlated to Analytic Reasoning (r-*61) and was negatively 
correlated to Antonyms {r^-.62). Function 2 was positively correlated to 
Reading Comprehension (r=.59). Students enrolling in this coursework improved 
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■Reading Comprehension (r=.59). Students enrolling in this coursework improved 
in Antonyms and Reading Cojnprehonsion but declined in their Analytic Reasoning 
abilities. 

Cluster #2 had high positive group mean on Function 1. Function 1 was 
positively correlated to Analytic Reasoning (r-.61) and was negatively 
correlated to Antonyms (r=^-.62). Students enrolling in this cluster gained in 
Analytic Reasoning but declined in Antonyms. 

Cluster #3 evidenced a high positive group mean on Function 4 and high 
negative group mean on Functions 3. Function 4 was positively correlated to 
Analytic Reasoning (r=.57) and Quantitative Comparisons (r=.50). Function 3 was 
positively correlated to Reading Comprehension {r=.64). Students taking Cluster 
#3 coursework improved in Analytical Reasoning and Quantitative Comparisons but 
declined in Reading Comprehension. 

Cluster #4 had high positive group means on Functions 2 and 5, and a high 
negative group mean on Functions 3. Function 5 was positively correlated to 
Quantitative Comparisons (r=-.67). Students enrolling in this cluster showed 
gains in Quantitative Comparisons. The results for Reading Comprehension were 
inconclusive. 

Cluster #5 had high negative group means on Functions 1 and 2. Students 
enrolled in this coursework gained in Antonyms but declined in Analytic 
Reasoning, and Reading Comprehension. 

Cluster #6 encompassed high negative group means on Functions 2 and 3, and 
a high positive group mean on Function 1. Students signed up for this 
coursework pattern declined in Antonyms and Reading Comprehension but gained in 
Analytic Reasoning. 

Cluster #7 had high positive group means on Functions 1 and 5. Students 
taking thib coursework pattern gained in Analytic Reasoning and Quantitative 
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Comparisons and declined in Antonyms. 

Cluster #8 consisted of three courses. Two courses were misclassif ied. 
Therefore, no further analysis was conducted with this cluster. 

Cluster #9 had high positive group means on Functions 1 and 4. Students 
enrolled in these courses improved in Reading Comprehension, Analytic Reasoning, 
and Quantitative Comparisons. 

Cluster ilQ had high negative group means on Functions 2 and 5, and a high 
positive group moan on Function 3. Students enrolling in these clusters showed 
declines in Quantitative Comparisons. The results for Reading Con^rehension 
were inconclusive. 

Cluster #11 encompassed high positive group means on Functions 4 and 5. 
Students registering in this coursework gained in Quantitative Comparisons and 
Analytic Reasoning. 

Cluster #12 had a high positive group mean on Function 3. Students taking 
courses in this cluster improved in Reading Comprehension, 

Cluster #13 consisted of three courses. Two courses were misclassif ied» 
Therefore, no further analysis was conducted with this cluster. 

Fiture 7-12 indicates that for the Georgia State group. Functions 1 to 5 
explain 93,83% of the variation in residuals. Lambda values were again 
significant at the .0001 level- Functions 6 to 9 individually account for less 
than b percent of the variance. Thus, only Functions 1 to 5 were used in the 
analysis of the coursework clusters. Since these functions were correlated with 
Heading Comprehension, Quantitative Con^arisons, Analytic Reasoning and 
Antonyms, it suggested that these GRE item- type residuals were predeominant in 
explaining the coursework patterns of the Georgia State group • 

It should be cautioned that the association was established at the cluster 
level. No direct causal link is intimated between student enrollment in any one 



- 170 - 

18S 



•^ijy students who enrolled in th€-se courses had higher residuals. The cluster 
serves to hypothesize relationships between coursework patterns and the general 
learned abilities measures by the item-types of the GRE. One can say that 
students who enrolled in specific patterns of coursework tended to iitprove on 
specific GRE item-types, while others who enrolled in different coursework 
patterns did not tend to show such gains. This evidence affirms the hypothesis 
that student gains in general learned abilities are associated, positively and 
negatively, with the coursework in which they enrolled. Further analysis is 
required to determine the nature of these associations. 



Summary 

The examination into the subsamples of the Clayton State and Georgia State 
students was the focus of this chapter. The goal was to determine whether the 
assumption underlying common course numbering schemes in statewide public higher 
education held validity. In short, did taking coursework at the community 
college (Clayton State) produce the same effect as taking comparably-numbered 
coursework at GSU. The patterns of coursework for Georgia State and Clayton 
State identified in this project were logical and salient to the extent that the 
group analyzed was homogeneous in its gains in general learned abilities. If 
all undergraduates were to benefit from a single of general education coursework 
requirement— regardless of insitution enrolled--the cluster analysis would 
produce such a core among all such coursework taken. This in fact did not 
occur. Logical sets of courses were found among the different groups of 
students, while the cluster resulting from the analysis of the total sample was 
less discrete and logical. The results did not support the efficacy of a 
statewide core curriculum and common course numbering system. Only forty 
percent of the courses enrolling 5 or more students were part of the general 
education requirements and associated with gains in the Clayton State student's 
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education requirements and associated with gains in the Clayton State student's 
learning. Seventeen percent of the courses enrolling 5 or more students were 
part of the general education requirements and associated with improvement in 
the Georgia State student' b learning. Such a finding argues against the 
establishment of a core curriculum as advocated by the National Endowment for 
the Humanities (1989). The results support the view of the advocates for 
distributive requirements in general education since there were differences in 
the gains these students demonstrated in student incoming abilities^ general 
learned abilities, and differences in coursework patterns in which they 
enrolled. In general, community college students showed greater gains than did 
Georgia State, took a more discrete set of courses and from a more limited array 
of choices. Thus, our support the current use of a wide range of options in a 
distributional general education requirement. Instead, it suggests that 
discrete arrays of coursework be identified which are more appropriate and 
productive for different ability levels of students. This conclusion was 
manifest in the findings of the analysis of Clayton State and Georgia State 
students. Discrete sets of coursework were identified that were beneficial to 
these students. These results suggest the need for greater academic advising in 
\Uidergraduate course selection or greater prescription in the curriculum. The 
cluster analytic model also can be used to identify coursework which has been 
beneficial to students of specific ability levels, interests and aptitudes 
(3ones & Ratcliff, 1990). 
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VI IT Sunroary and Conclusions 



"'hij> report concentrated on an examination of transcripts and aosessnient 
test scores from the combined sair^sXe of graduating seniors at Clayton State 
college in the 1981-88 and 1988'*89 academic years • The main purpose of this 
project was to determine if enrollment in different patterns of coursework were 
associated with gains in the general learned abilities of undergraduate 
students. The answer to this question was consistently "yes". Roughly 9 of 
each 10 courses analyzed were accurately grouped according to differential 
effects in the general learned abilities of students. Taking different patterns 
of coursework did lead to different types and levfils of development as measured 
by the nine item- types of the GRE General Test. 

Several consistent findings emerged from the analysis of coursework 
cluHters. First, the development of general learned abilities did not have an 
exact one-io-one relationship with departmental categories. All quantitative 
reasoning development did not occur exclusively in Mathematics classes^ for 
example. Consequently, simple counts of the number of credits or courses a stu- 
dent has taken in a particular subject may not be a reliable proxy of general 
learning in the attendant subject area. Quantitative skills may be developed in 
a variety of Business, Accounting, and Management courses as well. Second, the 
development of general learned abilities was not confined to the lower division. 
This finding was consistent for the combined sample at Clayton State College and 
Georgia State native sample groups. While there is clearly a difference between 
general education and general learning (a^ measured by the GRE), general 
education requirements should be re-examined in light of student improvement in 
general learned abilities. Coursework that students who showed significant 
gains took should be examined, evaluated and incorporated into the general 
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education sequence of the college. Third, beyond the college catalog, there was 
little formal monitoring and description of the curriculum in terms of general 
learned abilities at the college-wide or university-wide level. Colleges should 
regularly monitor the number of credits and courses in their curriculum. 
Without this baseline data, the extent to which students share a common learning 
experience at a college cannot be readily determined. 

The relationships established through the cluster analytic model were asso- 
ciational, not causal. Once a set of courses has been linked to score gains in 
a specific learned abilities, a targeted investigation can be launched to deter- 
mine the commonalities of teaching-learning environment, of student and faculty 
expectations of performance, of the specific abilities of the students who en- 
rolled in the classes. But regardless of what hypotheses are generated about 
why this coursework is associated with gains in learned abilities, one can state 
with confidence that students who enrolled in this coursework demonstrated gains 
on a specific type of learned ability. 

The coursework patterns associated with gains in general learned abilities 
varied in several ways. The coursework relationships varied according to the 
definition of the curriculum used. The relationship of patterns of coursework 
found in the analysis of courses taken by 5 or more students were different from 
those taken by 2 or more students. Also, the relationship between coursework 
and learned abilities varied from year-to-year. This variation in coursework 
was occasioned by the 5 percent per year variation in change in course 
offerings. This finding has broad implications for any institution-wide 
assessment of student learning. Courses are constantly changing. Student 
selections of those courses vary from year to year as well. 

Coursework patterns with negative means have limited meaning. A negative 
mean of residuals on a coursework cluster does not necessarily mean actual 
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decline in general learning, only decline about the group tested. Regression 
automatically defines half the residuals as negative. The mean performance of 
the group tested is the basis upon which the individuals' GRE item- type scores 
are predicted from the corresponding SAT scores • By definition, half the group 
falls below the mean, half is above it. Therefore^ those with negative means 
may have gained in general learned abilities. The negative sign shows that 
their gain fell below the mean of students in the sample. Declines in general 
learned abilities are relative to the sample group and may or may not represent 
actual declines in abilities. 

Gains in student learned ability may be attributable to learning outside 
the classroom. Cour; 9s in this analysis are the settings for analysis. 
Comparable analysis could be conducted according to student residential 
groupings. It is known, for example, that on many campuses the academic 
performance of students in one fraternity or sorority may be consistently above 
the average for that college, while students living at another Greek residence 
setting may be well below the campus norms for academic performance. Does 
living in a fraternity or sorority cau>:e higher or lower academic performance? 
Not necessarily. These are selective and self-selected residential situations; 
the relationship with academic performance is associationai. . 

The coursework patterns identified in this research include general 
education, major, minor and electives. No a priori distinction was made 
according to these categories prior to analysis of the data. A physical 
education course in Tennis may be associated with iug^roved learning in Regular 
Mathematics. This does not mean that enrolling in Tennis causes improved 
abilities in Regular Mathematics. What it does mean is that students who 
enrolled in this course tended to improve in this learned ability. Why? The 
cluster analytic model does not tell us why. Its purpose is to sort through the 
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hundreds or thousands of courses in the college curriculum. The model points 
out those coursework patterns taken by students who improve in general learned 
abilities. 

The cluster analytic nxjdel of analysis is admittedly complex. It would be 
simpler to calculate residuals on one item-type, such as Regular Mathematics, 
and then to rank order all coursework according to the mean residuals of 
students in each course. This would give a picture of each course according to 
one measure of general learning. However, it would not give an idea of the role 
of that strength of that ability or measure relative to other measures of 
general learning used in the assessment. The discriminant analysis shows the 
role of various types and measures of general learning relative to the 
coursework that students take in college. 

Where does this leave us? First, we acknowledge that the cluster analytic 
model performed well with the combined sample at Clayton State College. It 
sorted and classified coursework according to a given set of measures of general 
Ifcarned abilities. Second, some coursework patterns identified made sense. 
Some courses involving mathematics or business were associated with gains in 
mathematics abilities; many courses in mathematics and business were not. 

If all student coursework was distributed randomly throughout the 
curriculum, so too would be their test score residuals. A non-random 
distribution of residuals implies that specific coursework makes a difference in 
the development of general learned abilities. Only those courses selected by 
students showing improvement above the mean for their group should have positive 
mean course residuals. This research affirmed the differential coursework 
hypothesis. This research also affirmed the person environment fit hypothesis 
(Pascarella, 1985). 
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This* research also rejected the notion of a common core of coursework for 
the development of general learned abilities. However, it did not endorse or 
support the maintenance of the distributional forms of general education. 
Clearly, certain courses were more appropriate for students of lower academic 
ability, while other courses were associated with gains among higher ability 
students. This research suggests that while there may not be a perfect course 
of study for any one student, it is possible to link student assessment and 
transcript analysis to recommend an array of possible coursework that clearly 
has been associated with gains in general learned abilities. 

Student need not choose from several hundred or several thousand courses to 
have an effective education. The existence of a multitude of courses may be a 
healthy sign. It may show that colleges and universities accurately mirror the 
explosion and complexity of knowledge in the last decade of the twentieth 
century. Such complexity need not impair the development of general learned 
abilities in undergraduates. 

Yet, this research suggests that many students do not make wise course 
selections. At least they don't make smart choices in relation to those general 
learned abilities tested by the most commonly recognized post-baccalaureate test 
of general learning. The mean residuals of most courses taken by students in 
common with other students of the same cohort was near zero. This means that 
the general effect of coursework on student learning was randomly distributed 
across the curriculum. Perhaps only one- third of coursework taken in common (5 
or more students) could be found to have a positive relationship with general 
learning as measured. This rate of return can be improved by developing more 
discrete arrays of coursework applicable to general education requirements and 
by organizing and informing the student academic advising process so that 
students may choose among coursework aligned with their abilities and prior 
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learning. To that end, the cluster analytic model for identifying the 
differential effect of coursework holds promise to revising and. enhancing the 
conditions for excellence in undergraduate general learning. 

It is commonly assumed that the general curriculum leads to gains in 
general learning and the major * minor and elective curriculxam leads to gains in 
subject area learning. If such were the case, then the positive residuals on 
measures of general learning should occur with the courses which students hold 
in common. Also, negative mean course residuals should be primarily distributed 
among courses which students did not hold in common. 

We know a great deal about what colleges say should be the goals and 
standards for a baccalaureate degree. The DCP research suggests that much 
future research is needed to determine what curricular patterns and trends 
consistently produce the gains in general learning that institutions seek to 
impart to their students- The challenge of understanding the specific impact of 
coursework on the learning of students has just begun. 
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